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II.  INTRODUCTION 


The  research  to  be  conducted  over  the  four  year  period  of  this  contract  centers  around  three  areas  1) 
expansion  of  clinical  trials  capabilities  performed  under  an  ongoing  NIH  grant  and  their  continuation  after 
those  trials  are  completed;  2)  image  registration  using  the  mutual  information  cost  function  for 
evaluation/demonstration  of  image  compounding  and  sequential  examinations;  3)  approaches  to  real  time 
image-based  slice  positioning;  4)  the,  as  yet,  unique  effort  to  image  lactic  ducts  in  3D  and  perform 
preliminary  evaluation  of  their  diagnostic  potential.  Most  of  the  research  will  be  performed  in  parallel  by 
various  investigators.  The  following  time  line  for  the  third  and  fourth  years  was  given  in  the  funded  scope 
of  work. 


Table  1.  Time  Line  for  Present  and  Next  Project  Periods 

Year  3 

Year  4 

Clinical  Trials 

Validation/improvement  of  vascularity  measures 

Clinical  Trials  of  ~30  subjects 

-  +  -  + 

+  + 

Trials  data  analysis  and  write-up 

+  + 

Compound  and  Sequential  Imaging 

Develop  automated  image  registration  by  mutual  information 

+  -+  - 

+  ++  + 

Process  ultrasound  images  to  evaluate/demonstrate  compound  and 
sequential*  imaging  therewith 

+ 

Image-based  Slice  Positioning  [IBaR] 

Real  time  speckle  decorrelation  for  image  plane  positioning 

-  + 

Digital  ductography  development 

+  + 

Ductographic  analysis  of  selected  clinical  trial  cases 

+  + 

-  4- 

*Auto  registration  of  breast  images  from  multiple  exams  -i- =  concentrated  effort  -  =  modest 

effort 


The  following  series  of  deliverables  and  associated  materials  are  included  in  this  annual  progress 
report.  Included  are  a  listing  of  papers,  abstracts  and  manuscripts,  published  and  submitted. 

Year  03 

Item  1 :  Summary  report  of  clinical  trials  conducted  and  analyzed  in  the  third  year. 

Item  2:  Summary  report  on  initial  real-time  implementation  of  IBaR. 

Item  3:  Interim  report  on  ductography  techniques. 


IV.  BODY 

Progress  has  been  made  in  the  four  areas  of  emphasis  of  this  grant  as  listed  below. 

A.  Clinical  Trials  and  Vascularity  Measures 

1.  Clinical  Trials  of  ~30  subjects  (Deliverable  Item  1.) 

We  have  studied  7  subjects,  using  the  latest  apparatus  including  the  1.5  D  matrix  array  scanhead  for 
discrimination  of  benign  and  malignant  breast  masses  and  evaluation  of  3D  compound  imaging.  Some  of 
the  competing  grants  are  winding  down  and  the  rate  will  accelerate  in  year  04. 

2.  Trials  data  analysis  and  write-up 

This  contract  contributed  to  a  publication  and  a  poster  on  clinical  trials  methods  and  preliminary 
results.  References/ Appendices  1  and  5,  and  to  a  more  complete  clinical  trials  results  publication  about  to 
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be  submitted.  Some  of  the  outstanding  results  from  that  analysis  of  quantitative  3D  color  flow  imaging  for 
breast  cancer  discrimination  are  shown  in  Figure  1 . 

B.  Compound  and  Sequential  Imaging 

Because  of  our  exciting  advances  with  the  automatic  registration  of  two  or  more  3D  image  sets  for 
assessment  of  sequential  and  compound  imaging,  we  have  continued  to  emphasize  this  goal  during  year  03 
and  the  tasks  for  this  year  have  been  met  fully.  The  surprising  high  resolution  and  improvement  in 
detectability  of  low  contrast  regions  and  connective  tissue  layers  are  revealed  in  Figure  2. 

1.  Develop  automated  image  registration  by  mutual  information 

The  software  developed  here,  MIAMI  Fuse,  continues  to  be  refined  and  its  utility  for  application  to 
diagnostic  ultrasound  extended.  The  first  demonstrated  3D  coregistrations  of  ultrasound  image  sets  were 
published  in  Reference/ Appendix  2. 

2.  Process  ultrasound  images  to  evaluate/demonstrate  compound  and  sequential* 
imaging  therewith 

The  software  has  been  modified  and  the  registration  is  now  being  applied  on  patient  examples  for 
demonstration  of  each  of  several  uses,  including:  1)  tracking  changes  for  patients  undergoing 
chemotherapy  and  antiangiogenesis  therapy  (References/ Appendices  2&3);  2)  highlighting  and 

quantifying  changes  in  essentially  normal  breasts  as  a  prelude  to  improved  detection  of  preclinical  disease; 
3)  combining  3D  data  sets  from  various  ultrasound  imaging  modes  to  properly  register  the  unique 
information  provided  by  each  (References/ Appendices  2&3);  4)  expanding  the  field  of  view  by  combining 
several  partially-overlapping  3D  scans  to  see  the  full  volume  of  medium  to  large  lesions  and  reasonable 
amounts  of  surrounding  vascular  morphology  and  to  show  a  reasonable  volume  of  the  breast  for  tracking 
difficult  areas  of  the  breast  in  follow  up  studies  (Reference/ Appendix  3);  5)  combining  views  from 
multiple  angles  (compounding)  (Reference/ Appendix  4).  In  this  year  the  compounding  examples  have 
been  added  to  the  list  of  outstanding  results,  although  more  work  is  needed  to  address  the  effects  of  some 
artifacts. 


C.  Image-based  Registration  of  Slice  Position  [IBaR] 

1.  Real  time  speckle  decorrelation  for  image  plane  positioning  (Deliverable  Item  2.) 

As  part  of  our  research  on  this  topic,  we  have  continued  to  work  formally  with  one  company  and 
informally  with  another,  both  of  which  have  implemented  crude  real  time  versions  of  our  IBaR  technology 
(Reference  7).  One  has  implemented  software  which  uses  the  image  decorrelation  to  estimate  the  average 
scan  rate  over  the  entire  3D  sweep  and  applies  that  average  rate  to  assign  the  image  slice  spacings.  The 
other  does  not  appear  yet  to  apply  the  true  decorrelation  calculation  on  the  clinically-released  systems.  To 
accurately  estimate  elevation^  movement,  the  frame  rates  must  be  high  enough  such  that  consecutive 
images  are  not  completely  decorrelated.  We  have  arranged  availability  of  a  state  of  the  art  scanner  for  full 
time  research  that  has  the  real  time  frame  rates  (often  used  in  echocardiography)  that  are  required  for  more 
successful  measurement  of  scan  distances  at  normal  scan  rates. 

D.  Digital  ductography  development 

Software  tools  were  developed  in  the  previous  year  to  help  the  user  manually  find,  map  and  review 
ductal  structures  in  the  volume  of  B  mode  ultrasound  data.  Initially,  the  user  identifies  the  relative  location 
of  the  nipple  to  the  data  volume.  The  user  then  searches  the  volume  by  rotating  a  plane  about  the  nipple. 
When  a  duct-like  stmcture  is  seen,  the  user  draws  a  line  along  the  duct.  The  software  then  enables  him  to 
rotate  the  search  plane  around  this  line  to  verify  if  it  is  a  duct,  find  the  portions  of  it  not  in  the  initial  search 
plane,  and  identify  visible  lobules  feeding  it.  During  this  process,  the  user  can  mark  any  feature  as  part  of 
the  ductal  tree,  ultimately  creating  a  3D  mapping  of  ductal  structures  within  the  volume. 

1.  Ductographic  analysis  of  selected  clinical  trial  cases  (Deliverable  Item  3.) 

These  graphical  analysis  tools  have  proven  useful  for  visual  development  and  for  touchup  of 
automated  duct  definition  in  a  few  selected  cases.  The  main  work  will  be  in  more  automated  duct 
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development,  which  has  not  been  undertaken  extensively  due  to  the  major  emphasis  we  have  placed  on  the 
compound  imaging. 

We  have  developed,  in  conjunction  with  CIRS,  Inc.,  a  phantom  that  can  be  used  for  many  purposes, 
including  testing  of  detection  and  delineation  of  short  lengths  of  ducts.  This  CIRS  Model  44  cylindrical 
plug  phantom  is  shown  in  Figure  4.  It  contains  45,  high  contrast  stubby  cylinders  within  the  imaging 
range  of  the  Ml  2  scanhead  planned  for  this  study.  These  are  in  known  locations  relative  to  each  other  for 
tests  of  spatial  registration  accuracy  in  multiple  view  compounding,  as  well  as  for  the  ductographic 
detection.  This  and  any  other  phantoms  can  be  used  in  combination  with  the  beam  aberrating  plate  (Figure 
3)  to  produce  the  degraded  image  quality  found  in  breast  imaging. 


E.  Figures 


Age  (years) 


¥ 

c 

V 

tf? 

%il^ 

C 

e 


4» 

> 


(b) 


1 


0.75Ji‘ 


0.5. 


E  0.25^ 

40 

e 
a 


3 

H 


From  the  top  left,  the  curves  are: 


Not  shown: 


SWD,  Age,  and  GS  (Az  =  1.00) 
SWD  and  GS  (Az  =  0.97) 

SWD  and  Age  (Az  =  0.96) 
SWD  Alone  (Az  =  0.90) 

GS  Alone  (Az  =  0.89) 

GS  and  Age  (Az  =  0.92) 

Age  Alone  (Az  =  0.75) 
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False  Positive  Fraction  (1  -Specificity) 


Figure  1.  (a)  Discrimination  of  benign  (open  triangles)  and  malignant  (shaded  triangles)  breast  tumors 
for  38  cases  studied:  18  benign  (mean  age  46),  20  malignant  (mean  age  56).  (b)  Discrimination  Index 
including  one  subjective  measure,  grayscale  rating.  Points  are  spread  horizontally  for  clarity,  (c)  ROC 
curves  for  various  combinations  of  SWD,  age,  and  average  gray  scale  ( GS)  rating.  From  worst  to  best 
performance,  the  criteria  ranked  (5)  GS  alone,  (4)  SWD  alone,  (3)  SWD  combined  with  age,  (2)  SWD 
combined  with  GS,  and  (1)  the  combined  SWD-Age-GS  Index,  providing  %100  discrimination  in  this  38 
patient  population. 
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Figure  2.  (a)  Uncompounded  image  of  a  breast  from 

reference,  0° scan.  Statistical  analysis  showed  an  almost 
deal  increase  in  SNR  and  CNR  (contrast-to-noise 
^atio)  for  N  views  compounded  vs.  this  reference  view. 
Vote  the  overall  reduction  in  granularity  and  the  better 
delineation  of  connective  tissues  (gray  arrow)  in  (b). 


(b)  Same  image  area  compounded  from  5  scans  at  0°,  6.4  12.8^,  - 

5.1  and  -10.2°,  registered  using  11  control  point  warping 
transformations.  Most  importantly,  note  the  slightly  hypoechoic, 
lobulated  inhomogeneities  surrounding  the  left  end  of  the  hyperechoic 
structure  on  the  central  right.  The  white  arrow  denotes  one  such 
lobulation  with  a  thin,  echo  genic  rim  that  is  not  visible  on  any  of  the 
constituent  single  views. 


'c)  Uncompounded  image  of  a  breast  from  reference,  0  °, 
^can,  2.4  mm  (8  slices)  superior  to  image  (a).  Structures 
toted  in  (d)  are  not  seen  as  well  here. 


(d)  Same  image  plane  as  (c)  but  from  the  same  compounded  image 
set  as  (b).  Again  note  the  slightly  hypoechoic,  lobulated 
inhomogeneities  surrounding  the  left  end  of  the  hyperechoic  structure 
on  the  central  right  (e.g.,  white  arrows),  corresponding  to  those  at  the 
white  arrow  in  (b).  Note  also  the  improved  delineation  of  the  dark 
muscle  or  rib  in  the  lower  left  (gray  arrow)  and  the  tissue  striations 
extending  to  the  right  of  it. 


Figure  3.  CNC  design  for  the  mold  of  an  aberration  plate  introducing  phase  and  steering  errors  similar 
to  those  found  in  breast  tissue.  The  aberrator  will  be  60  by  100mm,  and  consist  of950m/s  silicone  rubber, 
filled  with  1540m/s  material.  The  undulations  are  about  ±1.1  mm  deep  (peak-peak)  and  include  beam 
steering  errors  as  well  as  defocusing. 


Figure  4.  Diagram  of  our  existing  CIRS  Model  44 
cylindrical  plug  phantom,  which  we  developed  jointly 
with  CIRS.  Within  the  5.5  cm  range  useable  for  the  GE 
M12  scanhead,  the  front  side  shown  contains  15  3  x 
6  mm  cylindrical  plugs  and  the  back  side  of  the 
phantom,  parallel  to  this  side,  contains  15,  1.5  mm  x  2.4 
mm  long,  and  15,  3  mm  x  3  mm,  cylindrical  plugs  of 
high  contrast,  —12  dB  re  the  background  material.  This 
yields  a  total  of  45  plugs  with  easily  localized  centers. 
Specific  attenuation  of  the  background  material  on  the 
back  side  is  0.5  dB  cm'^  MHzK  With  and  without  the 
surface  aberration  plate  of  Fig.  3-4.6,  this  phantom  will 
provide  excellent  measures  of  the  change  in  resolution 
(contrast  of  the  small  plugs)  and  increase  of  contrast  to 
noise  ratio  on  the  large,  low  contrast  cylinders. 
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V.  CONCLUSIONS 

Considerable  innovative  work  to  improve  and  evaluate  ultrasound  imaging  of  breast  cancer  has  been 
accomplished  with  primary  or  secondary  support  from  this  contract.  Several  of  these  advances  are 
receiving  national  attention  (References  6  &  8). 

Clinical  Trials 

The  3D  color  flow  quantification  and  other  advances  described  above  should  increase  the  sensitivity 
and  specificity  of  ultrasound  imaging  for  discrimination  of  benign  and  malignant  masses.  They  should 
also  improve  tracking  of  changes  during  treatment  or  even  detection  of  masses  by  highlighting  of  subtle 
changes  in  follow-up  studies.  Data  analysis  and  write-up  of  earlier  trials  has  been  completed  and 
contributions  were  made  to  advance  3D  image  acquisition,  display  and  quantification. 

Compound  and  Sequential  Imaging 

The  registration  achieved  so  far  is  not  at  pixel  level  yet  to  allow  accurate  pixel-by-pixel  correlation  of 
velocity  and  power  information  in  the  calculation  of  v*NPD,  but  the  registration  is  much  better  than  can  be 
done  with  visual  identification  of  corresponding  regions  of  interest  and  more  complete  delineation  of  gray 
scale  and  flow  structures  is  provided.  Fig.  2  reveds  the  success  of  techniques  required  to:  1)  use  the  3D 
registration  for  combining  information  from  several  types  of  ultrasound  scans  to  provide  an  image  volume 
of  improved  information  content;  2)  for  following  changes  in  the  patient  in  serial  exams.  Both,  but 
particularly  the  latter,  can  revolutionize  medical  imaging  and  interpretation.  The  former  is  approaehing  the 
quality  necessary  to  allow  compound  scanning  (imaging  the  same  tissues  from  different  viewing  angles)  in 
at  least  some  eases  and  combining  dedicated  high  resolution,  (multifocal)  gray  scale  images  with  f-CDI 
and/or  p-CDI  scans.  These  combinations  will  reduce  speckle  and  electronic  noise,  while  revealing  more  of 
the  structure  seen  only  at  a  few  of  the  image  look  directions.  The  combinations  also  will  allow  time  for 
higher  resolution  imaging  in  each  mode  while  combining  the  different  information  from  each.  The  types 
of  serial  exams  that  might  be  followed  include  tracking  of  tumor  size  and  vascularity  for  early  indications 
of  response  to  treatment,  as  well  as  serial  follow-up  or  even  screening  exams  to  reved  abnormal  growth  . 

Image-based  Slice  Positioning  [IBaR] 

The  possibility  of  practical  implementation  has  been  demonstrated  commercially,  albeit  in  a  reduced 
form  from  what  we  have  published  (References  7  and  9).  We  believe  it  will  be  practical  to  make  high 
quahty  3D  scans,  that  are  either  linear  or  sector,  but  possibly  not  both.  The  ability  to  make  accurate 
distance  measurements  will  probably  be  limited  to  fairly  short  distances  and  to  areas  around  relatively 
uniform  tissues. 

Digital  ductography  development 

The  functional  part  of  the  breast  is  the  glandular  tissue  which  is  found  in  15-20  lobes  which  radiate  out 
from  the  nipple,  j^ch  lobe  is  drained  by  its  central  duct  which  exits  from  the  nipple.  Almost  all  breast 
cancer  occurs  in  the  glandular  tissue,  and  a  significant  proportion  is  intraductal.  Therefore,  being  able  to 
find  and  map  the  ducts  might  greatly  increase  the  efficacy  of  B  mode  ultrasound  for  breast  cancer 
screening  and  diagnosis.  Display  of  the  ductal  tree  in  3D,  while  panning  through  the  gray  scale  images 
should  be  a  significant  aid  to  orientation  and  anatomy.  At  the  least  it  will  facilitate  registration  of  serial 
volume  image  sets  of  the  breast. 
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Abstract 

The  purpose  of  this  article  is  to  demonstrate  evaluation  techniques  for  three-dimensional  (3D)  ultrasound  (US)  imaging  and  their  potential  application  in  the  discrimination  and 
assessment  of  breast  cancer.  Our  ongoing  studies  include  women  with  mammographically  ob.served  breast  masses  scheduled  for  biopsy.  During  US  imaging  of  suspicious  breast 
lesions,  transducer  position  is  encoded  for  each  power-mode  and  frequency-shift  color  Doppler  image  and  for  each  standard  B-mode  image,  typically  90  images  in  each  mode.  Vessels 
are  then  displayed  as  reconstructed  color  volumes  in  3D,  superimposed  on  gray-scale  sections.  Radiologists  identify  suspicious  masses  and  rate  their  appearance  for  convention^ 
gray-scale  assessment  and  for  the  new  vascular  criteria.  In  an  initial  patient  population,  US  discrimination  of  malignant  masses  in  3D  volumes  demonstrated  superiority  over  their 
two-dimensional  counterparts.  In  the  related  patient  group  presented  here,  one  particular  vascularity  measure,  speed-weighted  pixel  density,  showed  statistically  different  {P  <  ,0()1) 
values  for  benign  (n  =13)  versus  malignant  (n  =15)  cases.  Given  the  authors’  modest  success  in  discriminating  breast  cancer  in  an  initial  patient  pool  with  use  of  other  vascularity 
measures,  results  of  this  second  patient  population  continue  to  suggest  vascularity  as  a  potential  cancer  discriminator. 
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Introduction 

Subjective  assessment  of  Doppler  ultrasound  (US)  imaging  may  be  enhanced  by  more-quantitative  analyses.  Displaying  vascularity  in  three  dimensions  is  relatively  simple,  since  the 
image  segmentation  is  perfomed  by  the  color  Doppler  flow  imaging  system,  and  sparse  vascular  patterns  are  easily  appreciated  in  three  dimensions.  A  number  of  investigators  (1-4) 
have  done  a  good  job  of  semiquantitative  evaluation  of  color  Doppler  flow  image  characteristics  of  the  breast;  however,  it  may  be  difficult  to  objectively  sample  the  sparse  breast 
vasculature  throughout  the  tumor  volume  in  totally  free-hand  two-dimensional  (2D)  sampling.  Stavros  et  al  have  achieved  promising  results  in  the  evaluation  of  breast  cancer  by  using  a 
niulti variable  gray-scale  as.sessment  of  2D  imaging  (5,6).  Cosgrove  and  Lees  also  indicated  that  the  addition  of  color  Doppler  flow  imaging  to  B-mode  imaging  improved  the 
diagnostic  accuracy  by  an  amount  approximately  equal  to  the  benefit  of  adding  US  to  mammography.  They  state  that  the  chance  of  malignancy  after  workup  must  be  less  than  2%  to 
consistently  avoid  biopsy  (7). 

Based  on  the  potential  benefiLs  of  combining  three-dimensional  (3D)  sampling  and  quantitative  assessments,  a  preliminary  study  was  performed  to  evaluate  which  Doppler  signals 
might  provide  discrimination  of  breast  cancer  from  benign  masses  and  to  compare  2D  and  3D  US  display  modes.  The  amount  of  blood  flowing  as  indicated  by  power-mode  color 
Doppler  imaging  (p-CDI)  (8)  and  the  velocity  indicated  by  mean-frequency  color  Doppler  imaging  (f-CDI)  both  lend  themselves  to  quantitative  measurement  (9).  To  evaluate 
information  provided  with  these  measurements  and  to  aid  in  visual  assessment,  we  undertook  a  visual  and  digital  analysis  of  information  in  a  study  that  included  3D  US  studies  of  20 
breajit  patients  scheduled  for  breast  biopsy.  We  have  subsequently  analyzed  studies  of  28  patients,  obtained  with  a  different  scanner.  Three-dimensional  breast  lesion  image  volume 
acquisition  techniques  and  visualization  schemes  are  presented  here  in  general,  and  a  particular  vascularity  measurement  (speed-weighted  pixel  density)  obtained  in  each  of  four  regions 
of  interest  (ROls)  in  and  around  the  mass  is  compared  with  standard  gray-scale  criteria  for  this  current  p^ient  pool. 
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Examples  of  Volume  Rendering 


Figures  2  and  3  are  taken  from  our  second^  and  current,  patient  population.  They  serve  to  demonstate  how  lesion  boundaries  can  be  defined  for  subsequent  quantitative 
calculations,  as  well  as  provide  an  appreciation  of  the  3D  gray-scale  and  and  vascular  renderings.  A  revealing  example  of  the  information  available  with  dynamic  3D  can  be  seen  in 
Figure  3.  In  our  chosen  coordinate  scheme,  x  represents  the  lateral  direction,  y  the  axial  direction,  and  z  the  elevational  direction.  Thus,  the  scan  plane  itself  can  be  considered  a 
constant  "z"  plane,  having  x-y  (lateral-axial)  coordinates.  Similarly,  the  plane  perpendicular  to  both  the  scan  plane  and  the  skin  surface  is  considered  the  constant  "x"  plane,  while 
the  plane  at  constant  depth  (C-scan)  is  the  ’’y"  plane.  Note  in  Hgure  3a  and  3b  the  excellent  section-thickness  (elevational)  resolution  obtained  with  this  matrix  array  (so-called  1 .5 
D)  transducer. 


c.  d. 


Figure  2,  Two-dimensional  and  3D  power  mode  display  of  an  elongated  infiltrating  ductal  carcinoma  in  an  81 -year-old 
woman,  (a)  Three-dimensional  view  demonstrates  two  planes,  one  of  which  is  the  original  scan  plane  and  the  second  an 
orthogonal  plane  that  depicts  the  extent  of  the  hypoechoic  mass,  (b)  Two-dimensional  image  of  the  original  scan  plane 
demonstrates  a  vessel  penetrating  the  mass,  (c)  Three-dimensional  image  with  an  ellipsoid^  ROI  defined  by  the  gray-scale 
borders  of  the  mass,  for  quantification  of  vascularity.  The  ROI  is  shown  as  a  translucent  green  ellipsoid,  to  depict  the 
approximate  extent  of  the  lesion,  with  the  penetrating  vessel  seen  as  red.  (d)  Three-dimensional  display  with  ROI  in  green  and 
surface  rendering  of  the  vascularity  in  blue.  (Click  any  image  to  view  its  full-size  version  [<100  kB  each])  . 
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Figure  3,  US  scans  of  a  SS-year-oId  woman  with  a  nonpalpable  mammographic  lesion  suspicious  for  cancer.  On  the  clinical 
mammogram,  the  suspect  region  appeared  as  a  1 .3-cm  macrolobular  circumscribed  mass  with  no  microcalcifications.  Final 
pathologic  studies  revealed  carcinosarcoma  (half  ductal,  half  chondrosarcoma),  (a)  Lateral-axial  plane  (scan  plane,  or  z  plane). 
The  smooth  ellipsoidal  borders  (approximately  1 .6  x  0.9  cm)  of  the  mass,  approximately  0.6  cm  from  the  surface,  could  cause 
the  reviewer  to  miss  the  clearly  malignant  features  in  other  views,  (b)  Axial -elevation  al  plane  (transverse  plane,  or  x  plane). 
The  shadowing  in  this  view  suggests  malignancy,  particularly  on  the  right  side  of  the  overall  mass  shown,  (c)  The  plane 
shown  here  is  arbitrarily  oriented  off  either  of  the  axes  of  the  scan  plane  or  transverse  plane,  and  is  slightly  tilted.  The  slight 
outcropping  and  shadowing  on  the  right  side  of  the  mass  further  suggests  a  malignant  portion  to  this  overall  hypoechoic 
region,  (d)  Lateral -elevation  al  plane  (C-scan  plane,  or  y  plane).  In  Siis  constant-depth  view,  unavailable  from  the  scanner 
itself,  note  the  malignant  multilobular  diffuse  appearance  of  the  mass,  (e)  The  tumor  shows  ROI  definition  in  3D  and  2D 
views,  (f)  Three-dimensional  view  shows  irregular  margins.  (Click  on  images  a-d  to  view  full-size  versions;  images 
e  and  f  link  to  Quicktime  movies  [<I  MB  each])  . 
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Measures  of  the  amount  of  blood  in  a  region  (NPO^^^)  and  estimate  of  perfusion  ([Vn,][NPD,^]),  mathematically  defined  in  Table  1  (where  max  refers  to  the  the  maximum 
value  calculated  in  the  four  regions  described  in  Table  2),  were  plotted  versus  average  gray-scale  rating  (1  =  normal  to  5  =  highly  suspicious)  and  were  evaluated  for  each  subject 
in  the  previous  study  (10).  Using  discriminant  analysis  to  classify  cancer  versus  benign  masses  and  insisting  on  sensitivities  of  1(X)%,  we  achieved  specificities  of  82%  and  90% 
for  and  ([Vjj,][NPDj^gj^]),  respectively,  when  combined  with  gray-scale  ratings.  These  specificities  were  higher  than  for  gray  scale  alone,  but  the  statistical  uncertainty  in 

these  numbers  is  large.  Our  second  patient  pool  is  aimed  at  adding  greater  statistical  significance  to  these  values  and  exploring  other  possible  quantitative  measures.  Figure  4 
demonstrates  one  such  measure,  speed-weighted  pixel  density  (SWD),  as  described  in  Table  1.  It  is  mathematically  equivalent  to  pixel  density  times  the  average  velocity  and  is 
thus  similar  to  ([V|^][NPDj^jyj])  without  the  power  weighting.  SWDj^^^  refers  to  the  the  maximum  or  peak  SWD  value  calculated  in  the  four  regions  described  in  Table  2.  As 
shown  in  Figure  4,  the  mean  value  of  log(SWD^^)  is  significantly  different  for  the  benign  versus  malignant  groupings  (P  <  .001  that  the  mean  values  are  equal). 
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Figure  4.  Plot  of  the  log(SWD^j^)  (ie,  the  maximum  SWD  value 
for  the  four  ROIs  described  in  Table  2)  for  benign  versus  malignant 
breast  lesions.  Mean  values  and  their  standard  deviations  are  shown 
for  the  two  poopulations.  As  demonstrated  by  t  test  results,  the 
probability  that  the  mean  values  of  these  subgroups  are  different  was 
statistically  significant  (P  <  .001). 


Comparative  Results 

Receiver  operating  characteristic  (ROC)  analysis  was  performed  on  the  SWD  measure  and  average  gray-scale  rating  using  ROCKIT  software  provided  by  the  University  of 
Chicago’s  Department  of  Radiology  (from  the  current  version  of  an  ROC  software  package  developed  by  Charles  E.  Metz,  PhD,  and  his  colleagues).  Figure  5  demonstrates  the 
predictive  similarity  between  the  SWD  measure  and  the  average  gray-scale  rating.  The  graph  shows  the  raw  data  and  maximum  likelihood  estimates  of  single  binormal  ROC  curves 
for  both  measures,  given  a  subject  pool  of  28  patients.  The  Az  v^ues  calculated  for  SWD  and  gray  scale  were  comparable,  0.88  and  0.84,  respectively. 


ROC  Analysis 


Figure  5.  Calculated  true-positive  fractions  versus  false-positive 
fractions  of  log(SWDjj^)  and  average  gray-scale  ratings  for  our 
28-patient  population.  Overlaid  are  smooth  calculated  ROC  curve 
estimates.  Although  the  area  under  the  curve  was  slightly  greater  for 
SWD  performance,  the  difference  in  the  two  areas  (Az[SWD]  =  0.88 
versus  Az[GS]  =  0.84)  was  not  significant.  GS  =  gray  scale. 
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Discussion 

Our  data  transfer,  processing,  and  visualization  tools  have  been  developed  to  the  point  where  full  3D  volumes  and  vascularity  surface  renderings  can  be  reviewed  within  an  hour  of  the 
US  exarnination  itself.  These  developments  have  facilitated  the  visualization  of  vascular  morphology  and  made  easier  scanning  through  entire  US  volume  sets  from  different 
perspectives.  In  addition,  we  have  provided  the  radiologist  with  a  straightforward  method  for  defining  reproducible  ROI  ellipsoids  within  a  given  US  volume,  which  facilitates  a  variety 
of  quantitative  measures.  As  demonstrated  by  means  of  ROC  analyses,  these  objective  measures  of  vascularity  (most  recently  SWD)  in  and  of  themselves  have  been  shown  to  be  as 
effective  as  subjective  gray-scale  measures  for  our  current  patient  population. 

Given  these  observations,  we  believe  that  vascularity  measures  should  improve  the  discrimination  of  benign  and  malignant  solid  masses  with  US  and  that  the  results  are  probably 
applicable  to  different  scanners.  Larger  numbers  of  subjects  and  improved  ease  of  performance  of  good  vascularity  examinations  are  now  needed  to  allow  good  judgments  of  the 
achievable  diagnostic  accuracy  and  cost-effectiveness  of  adding  vascularity  to  clinical  US  assessments  of  breast  masses. 

Ease  of  performance  is  well  on  its  way,  with  advances  occurring  rapidly  in  3D  scanning,  image  analysis,  and  image  display.  If  power-mode  and  color  Doppler  flow  measures  (NPD 
and  SWD)  each,  separately,  continue  to  perform  comparably  with  gray-scale  measures  and  almost  as  well  as  the  power-mode  and  color  Doppler  flow  combination  required  for 
lo8[(V^)(NPDj^jyj)]  as  demonstrated  in  our  earlier  report  (9),  then  one  of  the  two  color  Doppler  flow  3D  scans  can  be  eliminated  to  speed  up  the  entire  process.  Alternatively,  the  US 
system  manufacturer  could  supply  the  power  frequency  signals  simultaneously  with  sufficient  signal  dynamic  range  for  accurate  assessments  of  the  perfusionlike  log[(V^)(NPDjj^j^j^)] 
on  a  pixel-by-pixel  basis.  For  imaging  vessel  patterns,  little  would  be  lost  by  requiring  the  same  system  settings  for  both  frequency-shift  and  power  modes  in  a  single  image. 

In  these  studies,  gray-scale  readings  were  performed  by  a  radiologist  highly  experienced  in  breast  imaging  and  US.  If  vascularity  measures  on  simple  ROIs  can  perform  at  least  as  well 
as  gray-scale  assessments  for  the  experienced  reader,  the  potent!^  for  use  of  vascularity  measures  in  improving  routine  diagnosis  is  high. 

Conclusions 

Our  modest  success  in  discriminating  breast  cancer  in  an  initial  patient  pool  with  other  vascularity  measures  and  the  results  from  this  second  patient  population  continue  to  suggevSt 
vascularity  as  a  potential  cancer  discriminator.  Our  visualization  and  quantization  tools  should  continue  to  facilitate  this  process. 
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Abstract  — We  demonstrate  the  ability  to  register  easily  and  accurately  volumetric  ultrasound  scans  without 
significant  data  preprocessing  or  user  intervention.  Two  volumetric  ultrasound  breast  scan  data  sets  were 
acquired  from  two  different  patients  with  breast  cancer.  Volumetric  scan  data  were  acquired  by  manually 
sweeping  a  linear  array  transducer  mounted  on  a  linear  slider  with  a  position  encoder.  The  volumetric  data  set 
pairs  consisted  of  color  flow  and/or  power  mode  Doppler  data  sets  acquired  serially  on  the  same  patients.  A 
previously  described  semiautomatic  registration  method  based  on  maximizing  mutual  information  was  used  to 
determine  the  transform  between  data  sets.  The  results  suggest  that,  even  for  the  deformable  breast,  three- 
dimensional  full  affine  transforms  can  be  sufficient  to  obtain  clinically  useful  registrations;  warping  may  be 
necessary  for  increased  registration  accuracy.  In  conclusion,  mutual  information-based  automatic  registration  as 
implemented  on  modern  workstations  is  capable  of  yielding  clinically  useful  registrations  in  times  <35  min. 
©  1999  World  Federation  for  Ultrasound  in  Medicine  &  Biology. 

Key  Words:  Ultrasound,  Mutual  information.  Registration,  Volumetric,  affine.  Thin  plate  spline  warping. 


INTRODUCTION 

There  are  potentially  many  reasons  why  ultrasound 
examinations  might  benefit  from  registration.  Images 
from  different  transducer  positions  are  at  least  partially, 
if  not  fully,  uncorrelated,  and  summing  them  can  signif¬ 
icantly  improve  the  contrast-to-speckle  noise  ratio. 
Doppler  color  flow  acquisitions  suffer  from  undetected 
flow  where  the  flow  is  parallel  to  the  transducer’s  face. 
Volumes  collected  from  different  viewing  directions 
may  detect  such  missing  flow  vectors,  and  registration 
across  acquisitions  can  assist  in  computing  flow  velocity 
vectors  that  are  consistent  across  different  views.  Per¬ 
haps  the  most  important  reason  to  register  ultrasound 
data  sets  is  to  assist  the  comparison  of  serial  examina¬ 
tions  performed  on  the  same  patient.  Presenting  two- 
dimensional  images  from  different  serial  examinations 
with  the  same  orientation  and  partial  volumes  may  help 
the  clinician  more  easily  to  distinguish  what  has 
changed.  To  the  extent  that  echogenicities  from  different 
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examinations  can  be  approximately  equalized,  registered 
serial  examinations  can  be  subtracted  to  emphasize 
changes.  Such  differential  approaches  could  be  espe¬ 
cially  important  when  serially  monitoring  patient  re¬ 
sponse  to  chemotherapy  or  radiation  therapy. 

Although  such  automatic  registration  methods  for 
ultrasound  data  sets  may  be  desirable,  there  are  prelim¬ 
inary  reasons  for  caution  in  expecting  automatic  methods 
to  function  well.  Such  reasons  include  the  ubiquitous 
speckle  noise  and  attenuation  artifacts  such  as  “comet 
tails”  of  either  shadowing  or  enhancement.  Attenuation 
artifacts  that  present  differently  in  each  of  the  data  sets 
depending  on  transducer  orientation  may  compromise 
chances  of  successful  registration  for  any  automatic 
method.  An  additional  source  of  inconsistent  features 
comes  from  strong  coherent  echoes  that  are  observed 
from  structures  large  with  respect  to  wavelength  when 
observed  in  an  orientation  normal  to  the  reflecting  sur¬ 
face.  Thus,  coherent  echoes  as  well  as  shadows  in  dif¬ 
ferent  data  sets  are  often  sources  of  inconsistent  geom¬ 
etry  when  the  data  sets  were  acquired  using  different 
transducer  orientations.  Refraction,  i.e.,  ray  bending, 
may  yield  additional  inconsistencies. 
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Fig.  1 .  A  simplified  two-dimensional  example  of  the  registration  process  is  illustrated,  (a,  b)  Two  “multimodal”  images 
in  which  the  user  has  poorly  chosen  approximate  homologous  points,  (c)  Geometric  registration  that  maps  the 
corresponding  markers  onto  each  other  after  the  user’s  initialization,  but  before  automatic  optimization  begins.  After 
initialization,  although  the  markers  are  perfectly  aligned,  the  images  are  still  misregistered.  (d)  Resulting  final 
registration  obtained  by  the  optimizer’s  driving  the  locations  of  the  homologous  points  in  (b)  to  minimize  the  cost 
function,  —I,  />.,  maximizing  I.  (e)  Note  that  the  position  of  the  upper  marker  in  (b)  has  been  moved  by  the  optimization 
process  to  a  new  position.  (Reproduced  with  permission  from  Oxford  University  Press  from  Medical  Image  Analysis 

1997;3:195-206.) 


The  following  paragraphs  describe  the  use  of  a 
semiautomatic  registration  technique  that  appears  to  be 
capable  of  registering  ultrasound  data  sets  based  on  the 
preliminary  information  presented  herein.  The  term 
“semi”  qualifies  the  description  of  the  otherwise  totally 
automatic  method  because  the  initial  pose  between  the 
two  data  sets  must  be  set  by  the  user.  At  present,  the 
authors  are  unaware  of  any  semi-  or  fully  automatic 
registration  algorithms  that  have  been  applied  success¬ 
fully  to  ultrasound  data  volumes.  A  recent  review  of 
registration  techniques  in  general  can  be  seen  in  Maintz 
and  Viergever  (1998). 

METHODS 

Volumetric  ultrasound  data  were  acquired  using  a 
9-MHz  linear  array  clamped  to  a  linear  slide  and  position 
encoder  on  a  hand-held  frame  (Fenn  et  al.  1997).  The 
transducer  manually  was  swept  across  the  tissue  of  in¬ 
terest  in  a  direction  normal  to  the  linear  array’s  scanning 
plane.  Accuracy  of  beam  position  measurement  relative 
to  the  scan  frame  was  0.1  mm  in  the  focal  plane  of  the 
transducer.  Data  volumes  on  a  uniform  Cartesian  grid 
were  created  by  trilinear  interpolation  between  acquired 
images  from  known  positions. 

Pairs  of  such  ultrasonic  volumes  were  registered 
using  a  method  that  requires  an  initial  rough  guess  of  the 
orientation  between  the  two  data  sets.  As  can  be  seen 
later  in  the  Results  section  for  the  random  void  phantom, 
the  initial  orientation  may  be  far  from  accurate,  but  will 
still  within  the  capture  range  of  the  algorithm  and  result 
in  an  accurate  registration.  The  initial,  approximate  ori¬ 
entation  is  computed  by  having  the  user  position  a  min¬ 
imum  of  three  corresponding  point  pairs  in  each  data 
volume.  One  of  the  data  sets  is  chosen  by  the  user  as  the 
reference  data  set,  and  the  other  data  set,  which  will  be 
geometrically  transformed  onto  the  reference,  is  referred 


to  as  the  homologous  set.  Points  in  both  sets  are  depos¬ 
ited  by  clicking  the  computer’s  mouse  as  the  cursor  is 
positioned  manually  over  similar-looking,  three  features. 
Then,  under  automatic  control  of  the  optimizer,  the  first 
three  points  in  just  the  homologous  data  set  are  moved  to 
recompute  different  geometric  orientations  of  the  homol¬ 
ogous  data  set  with  respect  to  the  reference  data  set.  The 
points  in  the  homologous  data  set  are  referred  to  as 
control  points,  because  the  geometric  orientation  be¬ 
tween  the  two  data  sets  is  changed  to  satisfy  the  homol¬ 
ogy  defined  by  the  control  points.  The  automatic  opti¬ 
mizer  repeatedly  perturbs  the  loci  of  the  control  points  in 
the  homologous  data  set,  such  that  the  resulting  mutual 
information,  as  computed  for  all  voxels  in  the  reference 
volume  and  all  corresponding  voxels  in  the  homologous 
volume,  is  maximized  to  yield  the  final  computed  regis¬ 
tration  model  between  the  two  data  sets  (Collignon  et  al. 
1995;  Meyer  et  al.  1997;  Studholme  et  al.  1997;  Viola 
and  Wells  1995;  Wells  et  al.  1996).  Figures  1,  2,  and  3 
help  visualize  the  interplay  among  the  control  points,  the 
computation  of  mutual  information,  and  the  recursive 
optimizer  algorithm. 

The  entropy  of  a  data  set  is  defined  as  its  average 
information  content,  whereas  joint  entropy  is  the  average 
information  of  two  ordered  data  sets.  The  joint  entropy 
H(a,b)  of  two  data  sets,  a  and  b,  is  related  to  mutual 
information  of  both,  I(a,b),  by  the  following  classic  re¬ 
lationship: 

H(aM  =  H{a)  +  H{b)  ~  I{a,b)  (1) 

where  H(a)  and  H(b)  are  the  individual  entropies  of  data 
sets  a  and  b,  respectively  (Papoulis  1984).  As  can  be  seen 
in  the  classic  relationship,  mutual  information  I(a,b)  is 
the  amount  by  which  the  sum  of  the  individual  data  set 
entropies  must  be  reduced  to  account  for  correlations  that 
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under  current  geometric  assumptions  histogram  cell  modification 


Fig.  2.  Two-dimensional  joint  histogram  constructed  by  raster  scanning  through  all  voxels  in  the  reference  image  set  and 
incrementing  bin  counts  corresponding  to  the  gray-scale  values  from  geometrically  mapped  voxel  pairs  under  an 
assumed  geometric  transformation,  which  changes  with  each  iteration  of  the  optimizer.  (Reproduced  with  permission 
from  Oxford  University  Press  from  Medical  Image  Analysis  1997;3:195-206.) 


exist  between  the  two  individual  data  sets  (H  and  I  are 
non-negative).  The  previous  expression  defining  mutual 
information  in  relation  to  entropies  reduces  to  the  fol¬ 
lowing  equation  for  mutual  information  in  terms  of  the 
more  fundamental  probability  density  functions  for  data 
sets  a  and  b: 


I{ci,b)  —  \cladbp{ci,b)\og{p{aJy)lp{ci)p{b)).  (2) 


The  initial  geometric  transform  model  optimized 
via  moving  the  initial  three  control  points  is  the  simple, 
rotate-translate  (six  degrees  of  freedom)  transform.  In  a 
subsequent  optimization,  the  parameters  for  a  full  affine 


(linear,  12  degrees  of  freedom)  transform  are  estimated 
via  four  control  points  (the  three  previous  plus  one  new 
control  point  positioned  automatically  into  the  homolo¬ 
gous  data  set  using  the  previously  optimized  rotate- 
translate  transform).  Finally,  if  desired,  repeated  thin 
plate  spline  (TPS)  warpings  are  computed  using  five  or 
more  control  points  as  necessary,  where  the  previous  full 
affine  optimum  transform  is  used  to  instantiate  the  addi¬ 
tional  control  points.  In  this  manner,  the  mutual  infor¬ 
mation  is  optimized  at  each  stage  of  increasing  geometric 
model  complexity,  while  requiring  the  user  only  roughly 
to  estimate  approximate  geometric  homology  for  the 
three  initial  points.  The  additional  control  points  beyond 
the  first  three,  which  eventually  are  mapped  into  the 
homologous  data  set,  initially  are  placed  in  the  reference 


Fig.  3.  Functional  flow  chart  of  the  automatic  registration  algorithm  for  one  optimization  cycle.  The  process  is  repeated 
using  the  previously  computed  optimized  control  point  positions  as  the  new  starting  vector,  until  the  incremental  change 
in  mutual  information  between  cycles  falls  below  a  user  set  threshold,  typically  0.0002  bits.  (Reproduced  with 
permission  from  Oxford  University  Press  from  Medical  Image  Analysis  1997;3:195-206.) 
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Fig.  4,  (a)  Image  from  the  red  channel  of  a  color  flow  volume  breast  examination,  (b)  Pseudocolored  image  of  (a). 
Lowest  echogenicities  are  represented  by  dark  blue,  intermediate  echogenicities  by  green,  and  highest  echogenicities  by 
bright  red  hues.  Color  mapping  relationship  is  explicitly  demonstrated  in  the  attached  gray-scale  and  color  bars,  (c) 
Power  mode  data  corresponds  to  same  loci  as  the  color  flow  slice  of  a  following  registration,  (d)  Composite  image 
constructed  using  “screen  door”  transparency,  where  adjacent  pixels  alternate  between  the  color  and  gray-scale  sources 

of  (b)  and  (c). 


data  set  by  the  user,  without  identification  of  correspond¬ 
ing  positions  in  the  homologous  data  set. 

Uniformly  sampled  data  volume  sizes  are  approxi¬ 
mately  400  X  350  X  55-100  voxels  depending  on  the 
manual  acquisition;  voxel  volumes  are  typically  80  X 
80  X  200  jutm^.  The  registrations  computed  herein  use 
decimated  reference  data  sets  to  speed  computation.  Dur¬ 
ing  the  initial  use  of  the  rotate-translate  model,  only 
every  fourth  voxel  in  the  three  major  coordinate  direc¬ 
tions  is  used  in  the  reference  data  sets.  Each  optimization 
cycle  stops  when  the  controller  requests  control  point 
movements  of  less  than  some  user  chosen  criterion, 
selected  here  as  0.1  mm,  in  each  coordinate  direction. 
The  optimizer  is  the  standard  Nelder  simplex  algorithm, 
often  referred  to  as  “amoeba”  (Press  et  al.  1988).  Opti¬ 
mization  cycles  are  repeated  until  the  mutual  information 
increment  in  the  last  cycle  is  less  than  or  equal  to  another 
user  chosen  criterion,  selected  here  and  for  almost  all 
other  modality  pairings  as  0.0002.  After  computing  the 
optimal  forward  transform,  which  maps  the  homologous 
data  set  onto  the  reference,  the  high-resolution  mapping 
of  every  voxel  of  the  reference  data  set  onto  the  homol¬ 
ogous  is  computed  once,  using  the  inverse  transform. 


RESULTS 

The  first  registered  volumetric  data  set  pair  shown 
in  Fig.  4  consists  of  color  flow  and  power  mode  Doppler 
data  sets  acquired  serially  during  the  same  patient  exam¬ 
ination  visit.  A  three-dimensional,  nine  control  point  (27 
degrees  of  freedom)  thin  plate  spline  warping  was  com¬ 
puted  in  3 1  min  of  CPU  time  on  a  200-MHz  DEC  Alpha 
workstation,  model  3000/500x,  using  a  total  of  1552 
iterations  (where  each  iteration  consists  of  calculating 
the  geometric  transformation,  reconstruction  of  the 
mapped  homologous  gray-scale  volume,  joint  histogram, 
and  resulting  mutual  information)  .  The  red  channel  of 
each  data  set  pair  was  used  to  compute  the  geometric 
mapping.  Although  radiologists  commonly  avoid  the  use 
of  pseudocolored  images,  such  composite  images  assist 
in  visualizing  the  results  of  registration.  Display  of  the 
registered  data  sets  as  shown  in  Eig.  4d  enhances  visual 
differentiation  of  the  individual  image  sources  and  facil¬ 
itates  the  visual  assessment  of  registration  accuracy  at  all 
locations  in  a  single  image. 

Eigure  5  allows  the  reader  to  assess  the  accuracy  of 
the  volumetric  registration  over  a  large  volume  of  sup- 
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Fig.  5.  View  of  the  registered  color  flow  and  power  mode  volumes  visualized  on  three  orthogonal,  intersecting  cut  planes 
using  a  color  composite  rendering  method  that  approximates  the  screen  door  transparency  shown  in  Fig.  4d.  The 
observer’s  viewpoint  presented  here  is  one  of  looking  back  at  the  transducer  from  a  distal  position  deep  within  the  breast 
(top  of  the  image  is  still  patient’s  anterior).  The  cut  plane  whose  orientation  is  nearly  perpendicular  to  the  viewer  is  the 
same  image  as  that  shown  in  Fig.  4d,  where  the  underlying  gray-scale  image  represents  one  of  the  (high  resolution) 
acquisition  planes  of  the  color  flow  data  volume.  The  intersecting  cut  plane  across  the  top  of  the  volume  renders  the 
registration  at  a  constant,  shallow  depth  below  the  transducer  (C-mode),  and  the  remaining  cut  plane  right  of  center 
renders  a  slice  of  the  registered  volume  below  the  line  traversed  by  a  single  transducer  element  during  the  manual  sweep 
of  the  transducer.  The  color  bars  that  define  the  pseudocolor  conversion  for  the  reference  scan  is  visible  on  the  reader’s  left. 


port.  The  color  composite  display  computes  the  relative 
color  contribution  to  each  voxel,  based  on  a  linear  com¬ 
bination  of  the  intensities  of  the  voxels  of  the  two  reg¬ 
istered  input  volumes.  The  orthogonal  cut  plane  method 
of  display  was  chosen  because  of  its  clarity,  intuition, 
freedom  from  confusing  intervening  structures,  and  the 
ability  simultaneously  to  display  spatially  diverse,  three- 
dimensional  locations. 

Figure  6a  demonstrates  the  effects  of  the  computed 
geometric  warping,  by  applying  the  same  warping  to  a 
rectilinear  grid  occupying  the  power  mode  volume.  Fig¬ 
ures  6b-6d  show  views  of  the  deformation  from  the 
right,  front,  and  top,  respectively.  Note  that  the  major 
component  of  the  geometric  transform  is  a  simple  affine 


transformation,  as  best  seen  in  Fig.  6d,  where  the  volume 
has  been  sheered  to  the  left  as  a  function  of  distance 
away  from  the  observer,  i.e.,  transducer  position.  Upon 
closer  inspection,  a  nonlinear  warp  can  be  observed  in  the 
front  central  section  of  the  volume,  the  same  region  occu¬ 
pied  by  the  lesion,  indicating  that,  during  the  second  scan, 
the  lesion  moved  differently  than  the  surrounding  tissue. 

The  second  pair  of  registered  volumetric  data  sets 
presented  here  consists  of  two  power  mode  Doppler  data 
sets  acquired  on  the  same  patient  and  separated  by  45  d 
during  chemotherapy  for  locally  advanced  infiltrating 
ductal  carcinoma.  In  contrast  to  the  previous  case,  the 
registration  model’s  complexity  was  capped  at  the  level 
of  the  full  affine  (linear,  12  degrees  of  freedom)  trans- 
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Fig.  6.  (a)  Computed  geometric  warp  of  the  power  mode  volume  is  demonstrated  by  applying  the  same  warp  to  a 
rectilinear  grid  occupying  the  initially  undistorted  power  mode  volume,  (b,  c,  d)  Views  of  the  warped  deformation  from 
the  right,  front,  and  top  of  the  scanned  volume,  respectively. 


form  due  to  the  resulting  excellent  overall  accuracy,  as 
determined  visually.  Computing  this  transform  from  the 
initial  rough  estimate  required  909  total  iterations  over 
the  period  of  1L3  min.  As  in  the  previous  case,  the  red 
channel  of  each  pair  was  used  for  computing  the  geo¬ 
metric  mapping  and  display  of  results.  Figure  7a  shows 
three  orthogonal  cut  planes  through  the  registered  data 
set  volumes,  as  visualized  from  a  position  above  the 
transducer  looking  deeper  into  the  scanned  volume.  Note 
the  shear  (most  appreciated  on  the  plane  nearly  perpen¬ 
dicular  to  the  viewer)  required  to  register  the  data  sets. 
Figure  7b  is  a  single  slice  of  the  more  recent  data  set,  and 
Fig.  7c  shows  the  data  from  the  earlier  scan  correspond¬ 
ing  to  the  geometry  of  the  slice  shown  in  Fig.  7b  obtained 
after  volumetric  registration.  Because  the  two  cases  were 
performed  with  nearly  equivalent  depth-gain  compensa¬ 
tion,  equalization  of  the  echogenicities  of  both  scans  was 
trivially  achieved  using  linear  gray-scale  contrast  adjust¬ 
ment.  Figure  7d  demonstrates  the  signed  difference  be¬ 
tween  the  two  scans,  where  the  yellow  hue  represents 
differences  of  positive  sign,  the  blue  hue  represents  dif¬ 
ferences  of  negative  sign,  and  the  intensity  for  each  hue 
is  proportional  to  the  magnitude  of  the  difference.  Al¬ 


though  some  bright,  horizontal,  linear,  coherent  struc¬ 
tures  clearly  are  represented  in  the  difference  image,  a 
lower  contrast  echogenic  structure  also  is  visualized  (ar¬ 
row),  Le.,  present  in  the  earlier  data  set  but  missing  in  the 
later  volume. 

In  addition  to  the  two  breast  data  sets  presented 
herein,  we  include  results  from  scanning  a  commercially 
available,  random  void  phantom  in  two  different  orien¬ 
tations.  The  phantom  is  made  by  Computerized  Imaging 
Reference  Systems,  Inc.  (Norfolk,  VA,  USA)  and  con¬ 
sists  of  two  slabs  of  randomly  placed  spherical  voids 
surrounded  by  a  random  scattering  medium  having  an 
attenuation  of  0.5  dB  cm”^  MHz~^  One  slab  consists  of 
5-mm  diameter  voids,  and  the  other  consists  of  3-mm 
diameter  voids.  The  two  slabs  are  separated  by  a  voidless 
slab  consisting  of  the  same  random  scattering  medium 
used  to  surround  the  voids  in  the  other  slabs.  Scattering 
from  the  voids  averages  14  dB  down  from  the  surround¬ 
ing  random  scattering  medium.  Nylon  fibers  run  parallel 
to  the  slabs  at  depth  ranges  of  2  and  6  cm. 

One  volumetric  scan  acquisition  was  made  by  man¬ 
ually  translating  the  transducer  parallel  to  the  sides  of  the 
phantom  and,  thus,  parallel  to  the  wall  of  spherical  voids 
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Fig.  7.  (a)  Orthogonal  cut  planes  through  the  registered  power  mode  data  set  volumes  from  serial  examinations 
visualized  from  a  position  above  the  transducer  looking  deeper  into  the  scanned  volume.  The  pseudocolored  volume  was 
acquired  45  d  earlier  than  the  gray-scale  volume,  (b)  Single  slice  of  the  more  recently  acquired  data  set.  (c)  Data  from 
the  earlier  scan  corresponding  to  the  geometry  of  slice  (b)  after  volumetric  registration  has  been  implemented;  (c)  also 
is  visible  as  the  pseudocolored  component  in  (a),  (d)  Signed  difference  between  (b)  and  (c),  where  the  two  hues  represent 
differences  of  opposite  sign,  whereas  intensity  of  each  hue  is  proportional  to  the  magnitude  of  the  difference.  In  the  color 
bar,  a  difference  of  +  127  is  represented  by  bright  yellow,  a  difference  of  zero  is  black  (center  of  bar),  and  a  difference 

of  - 128  is  bright  blue. 


with  the  transducer  held  approximately  perpendicular  to 
the  phantom’s  surface.  The  second  scan  was  taken  by 
reorienting  the  translation  path  45°  to  the  initial  position 
using  a  draftsman’s  triangle.  Thus,  in  the  second  scan, 
the  transducer  remained  approximately  perpendicular  to 
the  phantom’s  surface,  whereas  the  path  of  the  translated 
transducer  crossed  the  wall  of  voids  diagonally.  The 
initial  poor  placement  of  the  starting  markers  was  inten¬ 
tional,  to  demonstrate  to  the  reader  how  severe  misreg¬ 
istration  of  features  appears  (Fig.  8a)  using  the  display 
technique  shown  in  Figs.  4  and  5,  where  blended  pseudo¬ 
color  and  gray-scale  scans  demonstrate  the  large  capture 


range  of  the  automatic  registration  algorithm.  Note  that 
the  bright  gray-scale  line  in  the  homologous  data  volume 
represents  the  same  nylon  fiber  in  the  phantom  as  the  red 
pseudocolor  line  in  the  reference  volume.  The  angular 
misregistration  is  readily  apparent  between  the  two  data 
sets.  Note  that  the  voids  appear  to  be  filled  with  higher 
level  scattering,  due  to  the  misregistration  of  the  two  data 
volumes. 

The  results  of  the  final  registration  are  presented  in 
Fig.  8b.  The  linear  nylon  fiber  in  the  homologous  data  set 
has  been  superimposed  with  the  red  pseudocolored  ref¬ 
erence  data  volume  fiber.  The  linear  fiber  superimposi- 
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Fig.  8.  (a)  Initial  pose  of  the  45°  diagonal  scan  of  void  phantom  in 
gray  scale  with  0°  reference  scan  in  pseudocolor  is  displayed  on 
three  orthogonal  planes  thi*ough  data  volume.  Note  the  filled 
appearance  of  the  voids  as  well  as  the  lack  of  alignment  of  the 
linear  nylon  fiber  in  this  initially  misregistered  pose.  The  reference 
data  set  is  shown  in  pseudocolor,  with  the  same  color  conversion 
relationship  used  previously,  (b)  The  same  display  of  reference 
data  set  shown  in  (a)  after  automatic  registration  of  the  volumetric 
45°  scan  with  the  reference  0°  scan.  Note  that,  after  registration, 
the  voids  have  more  contrast,  because  they  are  not  inappropriately 
filled  with  misregistered  scatterers,  and  the  nylon  fiber  in  the  45° 
scan  is  nearly  collinear  with  its  position  in  the  reference  volume. 

tion  is  nearly  perfect  to  the  viewer’s  right,  whereas  a 
slight  misregistration  can  be  observed  to  the  viewer’s 
left,  as  a  partial  separation  of  both  bright  gray  and  red. 
This  slightly  visible  error  correlates  well  with  the  com¬ 
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puted  angles  of  -1.0°,  -45.4°,  and  -0.5°  for  0^,  6y,  and 
0^,  respectively,  of  the  rotate-translate  approximation  to 
the  computed  full  affine  geometric  transformation  ma¬ 
trix;  we  would  expect  0°,  45.0°,  and  0°  for  a  perfect 
result.  The  mutual  information  of  the  initial  pose  was 
0.064  bits,  which  was  optimized  to  0.229  bits  in  17 
cycles  of  1025  total  iterations  under  the  rotate-translate 
geometry  model,  and  finished  at  0.383  bits  after  16 
additional  cycles  consisting  of  a  total  of  1 869  iterations, 
using  the  full  affine  geometry  model.  It  is  worth  noting 
that  simple  low-pass  filtering  of  the  two  data  sets  signif¬ 
icantly  speeds  the  convergence  of  the  registration  algo¬ 
rithm,  by  reducing  the  grand  total  of  all  iterations  to 
1200,  which  require  total  compute  times  <9  min  on  a 
433-MHz  DEC  Personal  Alpha  workstation. 

DISCUSSION 

Volumetric  registration  was  computed  in  all  cases 
using  a  full  affine  transform  as  the  first  endpoint.  If  the 
resultant  registration  was  judged  by  visual  inspection  to 
be  flawed  in  local  regions,  a  higher  degree  of  freedom 
geometry  model,  Le.,  warping,  was  computed.  Thus,  the 
thin  plate  spline  warping  of  the  data  set  associated  with 
Figs.  1  and  2  was  implemented  using  both  a  number,  Le., 
nine,  and  placement  of  control  points  that  resulted  in  a 
better  global  solution.  The  improved  accuracy  of  the 
registration  resulting  from  warping  was  determined  both 
visually  and  quantitatively  by  the  increase  in  the  final, 
optimized  mutual  information,  Le.,  a  significant  27% 
increase  in  I  was  observed  from  0.805  for  the  full  affine 
solution  to  0.827  for  the  nine  point  TPS  solution.  The 
presence  of  a  small  mass  with  different  elastic  modulus 
than  the  surrounding  tissue  is  suggested  by  observing  the 
nonlinear  warping  depicted  in  Fig.  3,  because  the  com¬ 
puted  tissue  deformation  indicates  a  local  rotation.  This 
rotation  could  have  been  in  response  to  the  overall  de¬ 
formation  caused  by  the  pressure  of  the  transducer  on  the 
skin  in  the  process  of  scanning. 

For  the  case  of  the  void  phantom,  although  the  error 
associated  with  0y  was  <0.5°  from  the  expected  value  of 
45°,  the  error  of  1  °  in  0^^  can  be  explained  by  our  inability 
accurately  to  mount  the  transducer  perpendicular  to  the 
plane  of  the  manual  translation  baffle.  Any  deviation 
away  from  90°  in  the  angle  subtended  by  the  rotation  of 
the  autoscanning  plane  of  the  transducer  into  that  of  the 
manual  scanning  baffle,  Le.,  0^,  will  require  compensa¬ 
tion  by  both  0^  and  0^,  depending  on  the  rotation  im¬ 
posed  on  0y.  Additional  registration  accuracies  using 
phantom  data  from  computed  tomography,  magnetic  res¬ 
onance  imaging,  and  positron  emission  tomography,  are 
available  in  Meyer  et  al.  (1997).  Although  no  prepro¬ 
cessing  was  necessary  for  the  phantom  data  to  obtain 
accurate  registrations,  a  >50%  decrease  in  registration 


Semiautomatic  registration  of  volumetric  ultrasound  scans  •  C.  R,  Meyer  et  al. 


347 


time  was  obtained  by  low-pass  filtering  the  input  data 
sets  before  registration.  Such  behavior  should  be  ex¬ 
pected,  because  low-pass  filtering  increases  the  long 
range  correlations  in  the  data  sets  and  smoothes  the 
resulting  cost  function,  such  that  convergence  can  be 
obtained  without  entrapment  by  many  local  maxima. 

Because  bright,  linear  structures  in  the  scanned  vol¬ 
ume  occur  when  the  transmitted  beam  is  perpendicular  to 
an  echogenic  surface,  a  lack  of  overlapping  alignment  of 
such  linear  structures  is  expected,  due  to  the  variation  in 
transducer  angles  used  to  acquire  the  different  volumes. 
Conversely,  good  registration  of  less  intense,  incoherent 
backscattering  structures  is  expected  and  should  be  used 
as  an  independent  visual  evaluation  tool  for  qualitatively 
judging  the  accuracy  of  the  registration. 

The  robust  behavior  of  the  previously  developed 
mutual  information  for  automatic  multimodality  image 
fusion  (MIAMI  Fuse)  registration  algorithm  (Meyer  et 
al.  1997)  for  these  pairs  of  same-modality,  /.e.,  ultra¬ 
sound,  data  sets  was  a  pleasant  surprise.  No  preprocess¬ 
ing  of  the  data  sets,  such  as  speckle  reduction,  was 
necessary,  and  the  resulting  registrations  were  of  excel¬ 
lent  quality  and  were  obtained  in  relatively  short  times. 
Even  the  demanding  task  of  warping  using  nine  control 
points,  i.e.,  27  degrees  of  freedom,  yielded  accurate, 
repeatable  registrations,  a  feat  even  the  authors  initially 
considered  as  unlikely. 
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Abstract 

Several  potential  uses  of  image  registration  and  new  display  techniques  in  3-dimensional  ultrasound  imaging 
are  demonstrated  in  three  examples,  representing  several  types  of  applications:  First,  the  gray-scale  information  of 
a  color-flow  image  set  was  coregistered  with  a  high  resolution  gray-scale  set  covering  the  same  region  of  interest. 
The  high  accuracy  achieved  in  this  registration  suggests  that  this  technique  can  create  improved  images  from 
combined  ultrasound  modes.  Second,  several  parallel,  partly  overlapping  gray-scale  scans  of  a  female  breast  with 
several  lesions  were  coregistered  and  combined  into  one  larger  volume.  Only  in  this  extended  field  of  view  could 
the  lesions  be  viewed  simultaneously  and  their  relative  sizes  and  positions  be  appreciated,  suggesting  the 
potential  of  better  detection  of  changes  in  breast  regions  being  followed  for  potential  cancer.  Third,  the  image 
registration  was  applied  to  a  series  of  ultrasound  breast  exams  for  cancer  therapy  assessment.  In  spite  of 
considerable  tumor  shrinkage  between  the  two  scans  obtained  several  weeks  apart,  the  coregistration  mapped  the 
dominant  features  in  the  region  of  interest  and  thus  facilitated  tracking  of  changes  over  time,  both  with  direct 
comparison  and  with  displays  that  highlighted  changes.  The  quality  of  the  semiautomatic  coregistrations  achieved 
and  the  significance  of  the  applications  demonstrated  here  indicate  that  image  registration  may  become  a  valuable 
tool  in  improving  ultrasound  diagnostics. 
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Introduction 

Image  registration  or  coregistration,  the  alignment  of  two  or  more  2-dimensional  (2-D)  or  3-dimensional  (3-D) 
images  of  an  object,  has  been  applied  to  medical  imaging  for  more  than  ten  years.  One  major  application  of 
medical  image  registration  is  fusion  of  morphological  images  such  as  magnetic  resonance  imaging  (MRI)  and 
X-ray  computed  tomography  (CT)  with  functional  studies  such  as  positron  emission  tomography  (PET)  and 
single  photon  emission  computed  tomography  (SPECT)  (1-7).  Other  applications  of  coregistrations  include 
motion  detection  (8,  9),  subtraction  imaging  (10,  11),  and  treatment  planning  (12,  13).  A  comprehensive  review  of 
different  approaches  and  techniques  in  medical  image  matching  can  be  found  in  (14). 

Until  recently,  ultrasound  image  registration  has  not  been  successful,  partly  because  of  the  small  2-D  field  of 
view  and  possibly  strong  tissue  motion  and  deformation  which  make  it  difficult  to  obtain  consistent  image  pairs 
for  coregistration  using  conventional  manual  scanning. 

With  the  advent  of  3-D  sonography  in  research  (15,  16)  and,  more  recently,  in  commercially  available 
ultrasound  scanners,  volumetric  ultrasound  data  have  become  available  and  can,  in  principle,  be  registered  like 
3-D  data  sets  from  other  imaging  modalities.  However,  the  speckle  noise  and  image  artifacts  usually  present  in 
ultrasound  images  and  the  strong  angular  dependency  of  the  apparent  brightness  of  specular  reflectors  challenge 
the  robustness  of  registration  algorithms. 

Apparently  the  first  successful  coregistration  of  3-D  ultrasound  scans  in  a  phantom  and  in  the  breast  was 
reported  in  ( 17),  using  registration  software  that  required  user  definition  of  homologous  points,  lines,  and  planes 
in  the  data  sets.  Semiautomatic  coregistrations  of  ultrasound  scans  were  achieved  very  recently  (18,  19)  using 
software  based  on  the  mutual  information  (MI)  of  two  data  sets.  The  same  software  was  used  in  this  study  to 
demonstrate  three  examples  of  potential  applications  of  coregistration  in  ultrasound  breast  imaging.  The  display 
possibilities  in  this  electronic  medium,  especially  the  movie  features,  should  allow  better  visualization  of 
registration  quality  and  assessment  of  errors  than  conventional  journals. 

The  gray-scale  image  obtained  along  with  the  color  information  in  Doppler  modes  usually  suffers  from  poor 
spatial  resolution  compared  to  the  high  resolution  obtainable  in  the  gray-scale-only  mode.  After  registration  of  a 
3-D  Doppler  mode  data  set  with  a  gray-scale  set,  the  color  information  can  be  superimposed  on  a  high  resolution 
gray-scale  background,  thus  extracting  the  best  of  both  imaging  modes.  Potentially,  this  allows  more  accurate 
diagnosis  and  higher  sensitivity  in  lesion  detection  by  displaying  flow  information  in  place  with  a  high  quality 
gray-scale  image. 

The  small  field  of  view  (FOV)  usually  covered  by  a  high  resolution  scanhead  may  be  insufficient  to  use  the 
full  capabilities  of  3-D  ultrasound  to  detect  and  track  changes  in  breast  regions  being  screened  or  followed  for 
possible  lesions  or  response  to  treatment.  Often  the  FOV  is  also  too  small  to  use  the  fiiU  advantages  of  3-D  in 
displaying  morphologic  distortions  of  normal  tissue  and  flow  surrounding  the  mass  and  differences  between  the 
possible  mass  and  a  statistically  robust  sampling  of  surrounding  tissues.  To  create  a  complete  or  more  complete 
image  of  the  breast,  the  small  volume  covered  by  a  single  scan  can  be  extended  by  repeatedly  scanning  the  breast 
in  parallel,  partially  overlapping  sweeps  which  can  then  be  combined  using  coregistration  of  the  overlap. 
Compared  to  a  commercially  available  2-dimensional  extended  filed  of  view  approach  (20,  21),  our  Extended 
Volume  Imaging  (EVI)  allows  much  faster  acquisition  of  3-dimensional  volumes  and  is  applicable  as  a 
post-processing  scheme  after  the  individual  scans  have  been  obtained. 

In  ultrasound  screenings  of  the  breast,  some  malignant  changes  are  manifest  as  changes  in  echogenicity, 
texture,  and  morphology.  Early  detection  of  these  changes  can  improve  survival  rates  (22).  Changes  in  a 
structure's  size  or  shape  can  also  aid  detection  of  a  malignancy  or  give  important  feedback  in  serial  ultrasound 
exams  for  the  assessment  of  response  to  cancer  therapy.  Coregistration  of  3-D  ultrasound  exams  can  facilitate  the 
temporal  tracking  of  changes  by  coregistering  the  suspect  regions  of  interest  imaged  at  different  times,  thus 
allowing  direct  comparison  of  the  'before'  and  'after'  texture  and  morphology. 
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Materials  and  Methods 

A  LOGIQ  700  ultrasound  scanner  (GE  Medical  Systems,  Milwaukee,  WI)  with  a  12  MHz  1.5-dimensional 
matrix  linear  array  probe  was  used  to  obtain  3-D  data  sets  of  the  breast.  The  probe  was  mounted  in  a  holder  that 
restricted  it  to  a  linear  motion  in  the  elevational  direction  (perpendicular  to  the  image  plane)  while  electronically 
encoding  the  linear  position  of  the  probe  in  that  direction.  A  more  detailed  description  of  the  data  acquisition 
scheme  can  be  found  in  (23). 

The  holder  was  placed  on  the  breast  and  the  transducer  slowly  moved  by  hand  over  the  region  of  interest.  The 
high  resolution  gray-scale  (GS),  frequency-shift  (f-CDI),  and  power-mode  color  Doppler  (p-CDI)  images 
obtained  were  then  interpolated  and  resampled  with  a  uniform  elevational  spacing  of  approximately  0.4  mm 
using  the  position  data  stored  along  with  the  images.  Display  and  registration  of  the  data  sets  were  performed 
using  A  VS/5  and  AVS/Express  (Advanced  Visual  Systems,  Waltham,  MA). 

The  3D  image  registration  software  MIAMI  Fuse  was  originally  developed  for  fusion  of  data  sets  from 
multiple  imaging  modalities  including  various  combinations  of  MM,  CT,  PET,  and  SPECT. 

To  define  the  approximate  relative  position  of  the  two  data  sets  to  be  coregistered,  three  control  points  have  to 
be  defined  in  each  set.  This  is  usually  done  by  assuming  the  images  were  obtained  in  the  same  location  and  in  the 
same  orientation  and  requires  very  limited  user  interaction.  The  algorithm  then  repeatedly  performs  geometrical 
transformations,  rigid  or  full  affine,  on  one  of  two  sets  and  maps  it  onto  the  second  set  until  the  best  fit  between 
the  two  sets  is  found.  Here,  the  mutual  information  (24,  25)  of  the  data  sets  is  used  as  a  cost  function  to  determine 
the  quality  of  a  fit.  With  more  control  points,  warping  transformations  can  be  performed,  but  that  was  not  done  in 
this  study.  For  a  more  detailed  description  of  the  algorithm  and  evaluation  of  its  performance  with  PET/CT, 
PET/MFd  and  SPECT/CT  multimodal  data  sets  see  ( 26 ).  More  details  on  the  application  of  this  image 
registration  software  to  3-D  ultrasound  data  sets  are  given  in  (19). 

The  registration  software  was  used  here  to  demonstrate  registration  of  3-D  ultrasound  scans  of  the  breast  for 
three  different  applications.  For  each  class  of  application,  one  typical  case  is  displayed  to  demonstrate  its  potential 
value  in  ultrasound  diagnostics. 

All  patient  data  were  acquired  after  informed  consent  was  obtained. 

1.  The  first  application  is  the  fusion  of  color  information  from  Doppler  imaging  modalities  with  gray  scale 
information  from  a  high  resolution  gray-scale  scan  of  the  same  volume.  For  this  purpose,  the  right  breast  of  a  forty 
year  old  female  patient  was  scanned  in  f-CDI  mode  and,  several  minutes  later,  in  high  resolution  gray-scale 
mode,  using  average  elevational  frame  spacings  of  0.43  mm  and  0.47  mm,  respectively.  The  gray-scale 
information  of  the  two  scans  was  then  coregistered  and  displayed  in  various  ways  to  evaluate  the  goodness  of 
match. 

2.  Extended  Volume  Imaging  is  the  second  application  demonstrated  in  this  study.  Three  parallel,  high 
resolution  gray  scale  scans  were  obtained  in  the  left  breast  of  a  65  year  old  woman  diagnosed  with  a  60  mm 
invasive  ductal  carcinoma.  The  relative  position  and  alignment  of  the  scans  is  shown  schematically  in  Figure  1. 
Starting  with  a  scan  in  the  center  of  the  region  of  interest  (B,  average  elevational  frame  spacing  0.32  mm),  the  4 
cm  wide  aperture  of  the  transducer  was  then  shifted  1  to  3  cm  medially  (A,  frame  spacing  0.29  mm)  and  laterally 
(C,  frame  spacing  0.37  mm),  respectively,  before  the  second  and  third  scans.  Thus,  3-D  data  sets  with  25%  to 
75%  overlap  (AB,  BC)  were  obtained.  Volume  registration  was  then  performed  on  the  overlapping  regions  to 
find  the  necessary  transformations  and  exact  relative  positions  of  the  data  sets,  mapping  scan  A  and  C  into  the 
reference  frame  given  by  scan  B. 

Note  that  our  hardware  does  not  encode  the  lateral  position  of  the  transducer,  and  that  image-based  methods, 
such  as  our  volume  registration,  are  preferable  to  hardware-encoded  position  since  the  image  may  be  distorted  by 
tissue  motion  or  refraction  artifacts.  These  distortions  would  be  undetected  by  an  external  position  encoder  and 
could  therefore  lead  to  discontinuities  in  the  extended  image. 


http://deepred.rad.med.umic^^  isHl/dcma  i  -evisionib/materials.html 


Materials  5/28/99  11:04  AM 

Three  parallel,  high  resolution  gray  scale  scans  were  obtained  in  the  left  breast  of  a  65 
year  old  woman  diagnosed  with  a  60  mm  invasive  ductal  carcinoma.  The  relative 
position  and  alignment  of  the  scans  is  shown  schematically  in  Figure  1.  Starting  with  a 
scan  in  the  center  of  the  region  of  interest  (B,  average  elevational  frame  spacing  0.32 
mm),  the  4  cm  wide  aperture  of  the  transducer  was  then  shifted  1  to  3  cm  medially  (A, 
frame  spacing  0.29  mm)  and  laterally  (C,  frame  spacing  0.37  mm),  respectively,  before 
the  second  and  third  scans.  Thus,  3-D  data  sets  with  25%  to  75%  overlap  (AB,  BC)  were 
obtained.  Volume  registration  was  then  performed  on  the  overlapping  regions  to  find 
the  necessary  transformations  and  exact  relative  positions  of  the  data  sets,  mapping  scan 
A  and  C  into  the  reference  frame  given  by  scan  B. 

Note  that  our  hardware  does  not  encode  the  lateral  position  of  the  transducer,  and 
that  image-based  methods,  such  as  our  volume  registration,  are  preferable  to 
hardware-encoded  position  since  the  image  may  be  distorted  by  tissue  motion  or 
refraction  artifacts.  These  distortions  would  be  undetected  by  an  external  position 
encoder  and  could  therefore  lead  to  discontinuities  in  the  extended  image. 


Figure  1.  The  diagram  shows  the  relative  positions  of  three  individual  scans  A, 
B,  and  C,  combined  into  one  extended  3-D  field  of  view.  The  three  scans  are 
approximately  parallel  and  overlap  at  least  25%  to  achieve  sufficient  accuracy  in 
the  coregistration.  Scan  B  serves  as  a  reference,  the  overlap  AB  is  used  to  align 
scan  A  relative  to  B,  and  similarly  BC  is  used  to  align  C  relative  to  B. 


3.  The  third  application  presented  in  this  article  is  the  registration  of  ultrasound 
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Results 


1.  Multimodal  Ultrasound  Imaging 

Figures  2  and  3  show  the  results  of  registering  a  3-D  f-CDI  volume  with  a  high  resolution  gray-scale  volume. 
The  two  image  volumes  were  acquired  sequentially  during  a  single  patient  visit.  The  forty  year  old  female  patient 
was  diagnosed  with  right  infiltrative  ductal  carcinoma.  Note  the  13  mm  spiculated  mass  which  appears  in  both 
Figures  2(b)  and  3  (white  arrow). 


(a)  (b)  (c) 

Figure  2.  Composite  representation  of  an  f-CDI  volume  coregistered  with  a  high  resolution  gray-scale 
volume.  The  high  resolution  gray-scale  images  are  pseudo-colorized  from  blue  (weak  echo)  to  red  (strong 
echo)  and  superimposed  onto  the  gray-scale  of  the  f-CDI  set.  The  scans  were  performed  minutes  apart,  (a) 
Three  orthogonal  planes  of  the  3-D  coregistered  volume.  The  white  labels  indicate  our  naming  scheme  for 
the  three  planes,  (b)  Axial-lateral  (z)  scan  plane.  The  white  arrow  points  at  the  13  mm  spiculated  mass,  (c) 
Axial-elevational  (x)  transverse  plane.  Click  each  image  to  view  its  full  sized  counterpart.  Click  here  for  a 
Quicktime  movie  (444  kB)  scanning  through  the  entire  volume  and  containing  the  frames  shown  in  (a)-(c). 
For  a  high-quality  version  of  the  same  movie  (1.2  MB),  click  here. 


In  Figure  2,  the  volumetric  data  is  shown  as  composite  images  of  the  f-CDI  and  the  high  resolution  gray-scale 
scans.  Figure  (a)  displays  three  orthogonal  planes  of  the  3-D  volume,  (b)  and  (c)  display  the  axial-lateral  and 
axial-elevational  planes,  respectively.  The  gray-scale  component  of  the  f-CDI  is  represented  with  a  "gray-scale" 
map,  while  the  high  resolution  gray-scale  is  colorized  by  the  blue-green-yellow-red  spectrum,  representing  dark 
to  light  echo  regions.  The  two  image  sets  are  then  superimposed  onto  each  other.  Note  the  overlap  of  features  in 
all  three  images  of  the  figure,  with  reds  and  yellows  overlapping  the  lighter  features.  To  view  a  Quicktime  movie 
(444  kB)  scanning  through  the  entire  volume,  click  here  or  follow  the  link  in  the  figure  caption. 

In  Figure  3,  the  particular  scan  plane  shown  in  Figure  2(b)  is  presented  in  a  different  way.  In  this  case,  the 
coregistered  planes  are  not  combined  into  a  composite  view.  Rather,  a  diagonal  line  separates  the  gray-scale  from 
the  f-CDI  (upper  left)  from  the  high  resolution  gray-scale  image  (lower  right).  The  viewer  is  then  able  to  see  the 
alignment  of  various  features  at  the  diagonal  boundary  for  comparison.  Note  that  the  sharp  borders  of  the 
spiculated  mass  (white  arrow)  clearly  visible  in  Figure  3(b)  are  barely  discernible  in  the  gray-scale  from  the 
f-CDI  image  shown  in  Figure  3(a).  In  a  Quicktime  "wiper"  movie  (348  kB),  the  viewer  can  appreciate  both  the 
accuracy  of  the  registration  as  well  as  the  sharpness  of  the  high  resolution  gray-scale  over  that  of  the  f-CDI  view 
by  sweeping  the  border  back  and  forth  over  the  suspicious  region.  Again,  click  here  for  the  movie  or  follow  the 
link  in  the  caption. 
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(a)  (b) 

Figure  3.  Lateral-axial  scan  plane  shown  in  Figure  2(b).  In  both 
images,  the  left  of  the  diagonal  line  shows  the  gray-scale  from  the  f-CDI 
while  the  lower  right  shows  the  high  resolution  multifocal  gray-scale 
image,  (a)  More  of  the  lower  resolution  image  is  shown.  Note  the 
alignment  of  various  structures  at  the  diagonal  image  boundary,  (b)  In 
this  view,  more  of  the  high  resolution  gray-scale  is  shown.  Note  the 
detectability  of  the  spiculated  mass  (white  arrow)  just  to  the  right  of  the 
diagonal  line  in  the  upper  portion  of  the  image.  In  (a)  this  structure  is 
barely  discernible.  Click  either  image  to  view  its  full  sized  counterpart. 
Click  here  for  a  full  Quicktime  "wiper"  movie  (348  kB)  containing  the 
frames  shown  in  (a)  and  (b). 


Even  if  the  mapping  of  the  two  image  sets  displayed  above  appears  to  be  very  accurate,  close  inspection  of  the 
data  reveals  a  small  remaining  error  in  the  alignment  of  discernible  landmarks.  The  main  causes  of  error  appear  to 
be 

1.  Local  tissue  motion  and  deformation  as  caused  by  respiratory  motion  and  compression  of  superficial  tissue 
by  the  transducer  itself.  These  local  deformations  can  be  corrected  only  in  part  by  the  global  geometric 
transformation  applied  here  to  register  the  two  data  sets. 

2.  Noise  in  the  image  sets  or  any  other  inherent  differences  between  the  sets  to  be  registered  as  introduced  by 
employing  different  imaging  modes  or  modalities  (here:  gray-scale  and  f-CDI  ultrasound). 

3.  Changes  in  look  direction  which  cause  changes  in  the  apparent  brightness  of  specular  reflectors  since  the 
echo  received  from  specular  reflectors  strongly  depends  an  the  angle  of  incidence  of  the  ultrasound  pulse.  Even  if 
very  apparent  to  the  eye,  this  may  actually  only  be  a  minor  cause  of  misalignment  since  the  volume  covered  by 
specular  reflectors  is  small  compared  to  the  total  volume,  and  thus  the  mutual  information  affiliated  with  it 
contributes  little  to  the  maximization  of  the  total  mutual  information. 

The  quality  of  the  registration  was  evaluated  by  a  single  observer  as  described  above.  The  mean  alignment 
error  obtained,  based  on  1 3  manually  defined  common  landmarks,  was  1 . 1  mm.  It  should  be  noted  that  this  value 
depends  on  the  position  and  distribution  of  the  selected  points  throughout  the  volume  and  is  somewhat 
subjective.  The  peak  alignment  error  estimate  was  2.9  mm  on  the  left  side  of  the  image  near  the  chest  wall  whose 
movability  is  unrelated  to  the  transducer  pressure  applied. 


2.  Extended  Volume  Imaging 

Figure  4  displays  the  extended  3-D  field  of  view  of  an  invasive  ductal  carcinoma  scanned  in  a  65  year  old 
woman.  The  orientation  and  relative  position  of  the  three  scans  used  to  form  this  extended  volume  was  described 


http.7/deepred. rad.med.umich.edu/RSNA_EJ  revisionib/results.html 


Results 


Page 


Tuesday,  May  25,  1999 

'in  Figure  1.  In  this  particular  example,  the  central  (B)  and  medial  (A)  scans  overlapped  about  75%,  the  central 
and  lateral  (C)  scans  overlapped  25%,  resulting  in  a  total  width  of  the  extended  field  of  about  7.9  cm.  In  Figure  4, 
the  boundaries  between  the  displays  of  the  coregistered  scans  at  0.3  and  0.6  of  the  total  width  of  the  extended 
volume,  marked  by  white  arrows  in  the  2-D  view  of  Figure  4,  are  hardly  noticeable.  Note  that  no  averaging  or 
other  processing  (other  than  slight  image  compression  for  on-line  display)  was  employed  at  the  boundaries  to 
smooth  out  the  transition.  The  display  changes  abruptly  from  scan  A  to  B  and  from  B  to  C,  which  makes  the 
continuous  appearance  of  the  extended  view  even  more  remarkable. 

A  Quicktime  movie  (548  kB)  can  be  displayed  for  appreciation  of  the  quality  of  the  composite  view 
throughout  the  volume. 


Figure  4.  (Top)  3-D  view  of  a  volume  combined  from  three 
parallel,  partly  overlapping  scans  in  a  65  year  old  female  with 
invasive  ductal  carcinoma.  (Bottom)  2-D  view  of  the  axial-lateral 
plane  displayed  in  the  volume  above.  The  width  of  this  extended  view 
is  7.9  cm.  The  boundaries  between  the  original  scans  at  about  one 
third  and  two  thirds  of  the  width  of  the  composite  image  (white 
arrows)  are  hardly  noticeable.  Click  here  for  a  Quicktime  movie 
(548  kB)  displaying  an  elevational  sweep  through  the  extended 
volume.  For  a  high-quality  version  of  the  same  movie  (1.3  MB),  click 
here. 

In  this  example,  commercially  available  high-resolution  transducers,  with  aperture  widths  of  typically  four  cm. 
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could  have  only  fully  displayed  one  of  the  masses  at  a  time.  This  limitation  may  make  the  visualization  and 
evaluation  of  the  entire  structure  using  ultrasound  more  difficult.  In  the  extended  view,  the  three  distinct  masses 
are  displayed  simultaneously  in  their  relative  positions  as  determined  by  the  coregistration  in  the  overlapping 
regions.  This  allows  immediate  perception  of  the  entire  region  of  interest  and  reveals  more  of  the  overall  anatomy 
and  anatomical  relationships.  The  large  FOV  should  also  diminish  the  danger  of  missing  important  details  at  the 
edges  and  facilitate  measurements  within  the  extended  volume. 

The  remaining  misalignment  of  the  registered  scans  in  the  overlapping  regions,  based  again  on  visual 
inspection  of  identifiable  structures  and  measurement  of  their  apparent  shift,  was  estimated  to  be  0.8  mm  mean 
(1.5  mm  peak)  in  the  75%  overlap  region  and  1.4  mm  mean  (3.1  mm  peak)  in  the  25%  overlap  region. 


3.  Sequential  3-D  Imaging 

Figure  5  shows  an  example  of  a  coregistration  of  sequential  scans  of  a  breast  lesion  in  a  patient  undergoing 
CAF  chemotherapy.  The  65  year  old  woman  had  a  palpable  2.5  cm  mass  and  was  later  diagnosed  with  left 
invasive  ductal  carcinoma  with  a  focal  mucinous  carcinoma  component.  The  two  scans  displayed  here  were 
acquired  8  days  before  and  29  days  after  the  start  of  the  chemotherapy.  Two  more  scans,  obtained  and 
coregistered  55  and  78  days,  respectively,  after  the  start  of  the  therapy,  yielded  similar  registration  results  and  are 
not  displayed  here. 

Although  the  size  and  shape  of  the  lesion  changed  significantly  between  the  first  two  scans,  the  registration 
procedure  aligned  the  lesions  well  and  allows  better  comparison  and  visualization  of  changes  than  without 
registration. 
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(c)  (d) 

Figure  5.  Coregistration  of  two  sequential  scans  of  a  breast  lesion  in  a  patient  undergoing 
chemotherapy,  (a)  Central  B-scan  frame  of  the  first  scan,  obtained  one  week  before  the 
start  of  the  therapy,  coregistered  and  transformed  to  map  onto  the  second  scan,  (b) 

Corresponding  frame  of  the  second  scan,  obtained  5  weeks  later,  (c)  Combined  view  of 
the  same  frame  in  the  first  (left)  and  second  scan  (right).  Click  here  for  a  Quicktime 
"wiper"  movie  (304  kB)  showing  the  temporal  changes  throughout  this  frame  by  moving 
the  border  between  the  scans  from  left  to  right,  (d)  Pseudo-color  image  of  the  differences 
between  the  two  scans,  showing  increased  echogenicity  (blue)  where  the  mass  shrank 
(white  arrow),  and  decreased  echogenicity  (yellow)  below  the  mass  (gray  arrow).  Click 
here  for  a  292  kB  movie  displaying  the  differences  throughout  the  scanned  volume. 

Figure  5(a)  and  (b)  show  corresponding  frames  from  the  center  of  the  first  and  second  scan,  respectively,  after 
coregistration  of  the  volumes.  In  this  case,  the  first  scan  was  transformed  using  a  full  affine  transformation  to  map 
onto  the  second  one.  In  a  serial  study  with  scans  obtained  manually  and  several  weeks  apart,  small  changes  in 
look  direction  or  positioning  of  the  transducer  are  unavoidable  even  if  the  sonographer  carefully  tries  to 
reproduce  the  field  of  view  of  a  previous  scan.  These  variations  can  make  direct  comparison  of  two  serial  scans 
more  difficult.  However,  if  the  two  scans  have  enough  common  information,  the  registration  software  can  detect 
these  changes  and  transform  one  set  accordingly.  Hence,  the  slight  rotation  of  the  image  in  (a)  can  be  understood 
as  a  correction  for  a  small  change  in  view  direction  between  the  two  scans. 

Figure  5(c)  shows  a  combined  view  of  the  frames  displayed  in  (a)  (left  half)  and  (b)  (right  half  of  the  image), 
which  reveals  the  change  in  size  and  other  characteristics  of  the  lesion.  Also  seen  is  a  small  error  in  aligning  the 
specular  reflectors  at  the  top  of  the  images,  assuming  the  same  tissue  planes  are  producing  the  specular  reflections 
in  the  two  scans.  This  near  field  error  is  most  likely  induced  by  a  local  deformation  of  the  tissue  by  the  pressure  of 
the  transducer  against  the  skin.  The  estimation  of  the  average  misregistration  in  this  serial  study  is  difficult  and 
prone  to  error  because  of  the  additional  effect  of  drug-induced  changes  over  time.  These  changes  cannot  be 
unambiguously  separated  from  the  other  causes  of  registration  error  as  discussed  above.  However,  estimating  the 
average  error  by  identification  of  corresponding  landmarks  in  the  two  registered  sets  well  outside  the  tumor 
volume,  a  mean  error  of  1.4  mm  (peak  2.2  mm)  was  obtained. 

For  better  visual  evaluation  of  the  registration  quality  and  change  in  tumor  size,  a  Quicktime  movie  (304  kB) 
can  be  viewed  by  clicking  here  or  on  the  link  in  the  caption  of  Figure  5(c).  In  the  movie,  the  border  between  the 
two  scans  as  displayed  in  (c)  can  be  moved  from  left  to  right,  thus  revealing  more  of  the  first  or  the  second  scan. 

In  Figure  5(d) ,  the  change  in  tumor  size  is  highlighted  using  a  pseudo-color  display.  By  simultaneously 
feeding  the  gray-scale  values  of  the  coregistered  first  scan  into  the  red  and  green  channels  of  a  RGB  (red,  green, 
blue)  display  and  the  gray-scale  values  of  the  second  scan  into  the  blue  channel  of  the  same  display,  a  difference 
image  is  created  with  a  color  map  from  yellow  (echogenicity  of  first  scan  larger)  through  gray  (echogenicities 
identical)  to  blue  (echogenicity  of  second  scan  larger).  The  saturation  of  the  colors  is  proportional  to  the 
difference  in  gray-scale  value,  i.e.  the  larger  the  ^fference,  the  stronger  the  color.  The  shrinkage  of  the 
hypoechoic  mass  causes  an  increase  in  echogenicity  on  the  mass  periphery  in  the  second  scan  compared  to  the 
first.  This  difference  is  clearly  visible  as  a  blue  rim  (white  arrow)  surrounding  the  proximal  half  of  the  mass  in  (d). 
The  distal  part  of  the  mass  shows  no  such  blue  rim  and  therefore  appears  to  be  unchanged.  This  may  be 
explained  by  the  surrounding  anatomy  which  can  cause  asymmetric  tissue  deformation  and  displacement  around 
the  shrinking  mass. 

The  other  prominent  feature  in  this  figure  is  the  yellow  area  immediately  below  the  tumor  (gray  arrow), 
indicating  an  area  of  decreased  echogenicity.  This  decrease  could  be  explained  by  a  mostly  blood-filled  mass 
which  has  lower  attenuation  than  the  surrounding  tissue  and  thus  causes  an  apparent  enhancement  below  the 
mass.  The  smaller  mass  in  the  second  scan  then  causes  less  enhancement  and  therefore  a  decrease  in 
echogenicity,  which  can  also  be  seen  by  comparing  Figures  5(a)  and  (b),  but  which  is  more  obvious  in  the  color 
display  of  (d). 

By  clicking  here  or  on  the  link  in  the  caption  of  Figure  5(d),  a  movie  (292  kB)  can  be  displayed  showing  the 
difference  image  of  (d)  for  47  frames  of  the  scanned  volume.  Note  the  shrinkage  of  the  mass  in  all  3  dimensions 
by  following  the  blue  rim  throughout  the  movie.  Quantitative  evaluation  of  the  entire  scans  using  polygons  to 
manually  define  the  border  of  the  lesion  in  each  frame  yielded  a  reduction  in  lesion  volume  of  approximately 
27%. 
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4.  Reproducibility  of  Results 

More  examples  have  to  be  registered  to  study  case-to-case  variations  in  accuracy,  and  factors  limiting  the 
function  of  the  registration  algoritluii.  So  far,  two  examples  of  multimode  registration,  five  extended  volumes,  and 
five  sequential  cases  have  been  studied.  By  visual  assessment,  the  two  multimode  registrations  were  of  similar 
quality.  Also,  four  of  the  sequential  cases  were  of  comparable  registration  quality.  In  a  fifth  case,  no  registration 
could  be  found.  Visual  inspection  showed  that  either  the  sonographer  had  not  scanned  enough  of  the  same 
volume  in  the  two  sequential  scans,  or  the  mass  had  disappeared  completely  in  response  to  treatment.  In  extended 
volume  studies,  four  out  of  five  cases  were  similar  to  the  one  displayed  here.  In  the  fifth  case,  the  overlap  was  less 
than  1  cm  and  no  acceptable  registration  was  found. 


[  Previous:  Materials  and  Methods  ]  [  Next:  Discussion  ] 
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Discussion 

The  examples  described  are  successful  and  significant  applications  of  image  registration  in  3-D  ultrasound. 
They  demonstrate  the  feasibility  of  a  new  technique  to  improve  ultrasound  diagnostics  and  potentially  open  a 
new  range  of  related  applications. 

Routine  coregistration  and  combination  of  image  sets  obtained  using  a  variety  of  different  ultrasound  modes 
or  using  ultrasound  and  a  second  imaging  modality  should  be  possible  with  the  same  or  only  slightly  modified 
version  of  the  currently  used  registration  software. 

In  ultrasound  multimode  imaging,  the  coregistration  should  allow  image  optimization,  i.e.  replacement  of  a 
lower  quality  component  of  one  image  set  with  the  higher  quality  component  of  a  different  set,  as  in  the 
substitution  of  high  resolution  gray-scale  data  for  the  lower  resolution  gray-scale  'background'  of  color  flow 
imaging  modes,  as  well  as  image  fosion,  i.e.  the  combination  of  images  displaying  different  physical  properties, 
which  increases  the  diagnostic  value  of  either  one  of  the  original  images  by  relating  one  physical  property  to 
another.  Note  that  the  reduced  quality  of  the  image  in  CDI  mode  compared  to  gray-scale  mode  is  due  to  natural 
timing  constraints  in  image  formation,  and  is  therefore  likely  present  to  some  extent  in  most  or  all  modem 
ultrasound  scanners.  The  difference  is  probably  more  marked  here  because  the  matrix  array  used  allows  more 
image  improvements  in  the  gray-scale  images  that  cannot  be  realized  in  CDI  modes  than  usually  possible  with 
1-D  arrays. 

Possible  modes  for  image  fusion  can  include,  but  are  not  restricted  to,  frequency-shift  and  power  mode 
Doppler,  harmonic  imaging,  and  new  modes  being  researched  and  developed  such  as  imaging  with  contrast 
agents  or  ultrasound  elasticity  imaging  ( 27,  28 ).  Tlie  quality  of  the  registration  will  depend  on  the  information 
content  of  the  images,  more  specifically,  the  mutual  information.  Where  only  gray-scale  information  was 
employed  for  the  registration,  the  accuracy  did  not  allow  an  exact  pixel  by  pixel  match,  but  the  registration  was 
far  better  than  can  be  done  manually. 

In  multimodal  imaging,  coregistrations  of  ultrasound  scans  with  MR,  CT,  and  other  3-D  data  sets,  or  with 
projection  images  such  as  mammograms  can  be  envisioned.  If  ultrasound  scans  are  obtained  in  the  same  or  a 
similar  geometry  as  for  a  mammogram,  a  coregistration  of  a  projection  of  the  ultrasound  scan  with  the 
mammogram  could  combine  the  most  inexpensive  and  commonly  used  modalities  for  breast  imaging.  This  would 
add  3-D  information  to  the  2-D  mammogram  and  allow  more  accurate  tracking  of  spots  identified  in  the 
mammogram  using  3-D  ultrasound. 

Extended  Volume  Imaging  is  a  method  to  overcome  one  of  the  major  disadvantages  of  ultrasound  imaging 
compared  to  most  other  modalities:  the  very  limited  field  of  view  and  unsystematic  coverage  of  the  region  of 
interest.  In  breast  imaging,  a  wider  field  of  view  should  be  particularly  desirable  in  following  suspicious  regions 
and  in  detection  of  preclinical  disease.  To  follow  in  3-D  the  morphology  of  a  large  lesion  and  its  effect  on  the 
surrounding  tissue,  a  larger  field  of  view  will  be  essential.  A  combination  of  Extended  Volume  Imaging  and 
multimode  (gray-scale  and  CDI)  imaging  would  even  allow  tracking  of  vascular  and  tissue  morphology 
simultaneously  in  high  quality  images.  Screening  of  women  at  very  high  risk  for  breast  cancer  is  beginning  to 
draw  more  interest  in  the  US  with  incidental  detection  of  malignant  foci  at  locations  removed  from  the  site  of 
suspicion  (29).  Accordingly,  Extended  Volume  Imaging  is  likely  to  increase  the  sensitivity  of  ultrasound  in  breast 
cancer  detection.  A  wider  field  of  view  will  also  help  in  covering  previously  scanned  sites  and  thus  determining 
changes  over  time,  either  in  screening  of  high  risk  groups,  tracking  suspicious  regions,  or  monitoring  response  to 
treatment.  While  radiologists  and  sonographers  can  reach  very  high  skill  levels,  it  is  often  difficult  for  some 
operators  to  assure  that  the  same  views  are  obtained  in  two  or  more  serial  studies.  Interpretation  is  also 
complicated  when  the  interpreter  has  not  personally  performed  the  exam.  Both  problems  could  be  reduced  using 
Extended  Volume  Imaging. 

The  accuracy  of  coregistrations  of  partly  overlapping  volumes  appears  to  depend  on  the  size  of  the  overlap.  It 
is  possible  that  the  larger  the  overlap,  the  more  common  stmcture  is  available  to  align  the  scans,  and  the  less  the 
registration  algorithm  is  potentially  affected  by  noise.  The  minimum  overlap  required  for  acceptable  registration 
accuracy  can  vary  with  the  information  content  of  the  data  sets.  However,  in  the  examples  studied  so  far,  a  1  cm 
(25%)  overlap  generally  produced  acceptable  registrations,  judged  by  visual  inspection. 

Coregistration  of  sequential  exams  may  enhance  manual  (visu^)  detection  of  changes  in  several  ways,  and 
may  eventually  allow  for  semiautomatic  or  automatic  volume  change  estimations.  First,  the  registration  corrects 
for  slight  changes  in  scan  position  and  direction,  and  thus  allows  direct  comparison  of  the  same  areas  imaged  at 
different  times.  This  may  resolve  ambiguities  in  the  actual  location  of  the  scans  and  may  also  help  detecting  subtle 
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changes  that  go  unnoticed  in  unregistered  scans.  Second,  registered  sequential  scans  enable  new  display 
techniques  that  emphasize  changes  between  scans,  such  as  the  difference  image  shown  in  Figure  5(d).  Using  this 
type  of  difference  image  may  enhance  and  accelerate  the  detection  of  changes,  and  it  may  ultimately  be  possible 
to  segment  regions  of  shrinkage  and  growth  for  rapid,  semiautomatic  volume  change  estimations.  Third,  if  the 
registration  is  predominantly  based  on  a  mass  and  its  boundaries,  and  if  non-rigid  linear  or  non-linear 
transformations  are  used,  the  registration  could  map  the  boundaries  at  one  point  of  time  to  the  boundaries  at 
another  point  of  time.  The  transformation  matrix  itself  then  contained  information  about  size  changes  and  could 
also  be  used  for  semiautomatic  volume  change  estimations.  Note  also  that  the  registrations  can  be  obtained 
semiautomatically  and  with  very  limited  user  interaction.  Manual  registration  in  3-D  by  visual  comparison  of  two 
volumes  may  be  possible,  but  is  a  time-consuming  and  tedious  process,  and  subject  to  user  variability. 

For  serial  exams,  the  Extended  Volume  Imaging  technique  might  be  combined  with  coregistration  of  scans 
obtained  at  different  times.  This  would  allow  the  visualization  and  quantification  of  changes  in  large  lesions  over 
time.  In  fact,  the  acquisition  and  composition  of  an  extended  field  of  view  might  prove  to  be  necessary  for 
tracking  rapidly  changing,  large  lesions  because  in  this  case  a  small  field  of  view  might  not  contain  enough 
unchanged  structure  (which  is  usually  found  well  outside  the  volume  of  the  mass)  for  the  coregistration  to  work 
properly.  If  routinely  used  in  serial  exams  of  women  with  high  risk  of  breast  cancer,  coregistration  could  help 
detect  early  changes  not  apparent  in  multiple  small  FOV  scans,  thus  increasing  the  chances  of  effective  treatment. 

For  validation  of  quantitative  measurements  made  in  coregistered  image  sets,  an  automated  error  estimation  is 
desirable.  The  quality  of  coregistrations  can  vary  strongly  with  the  available  mutual  information,  which  depends 
on  the  amount  of  shadowing,  specular  reflectors,  and  local  tissue  deformation  present  in  the  image  sets.  Manual 
evaluation,  as  in  the  examples  presented  here,  may  not  be  practical  to  assure  accuracy  of  quantitative 
measurements  of  dimensions  over  long  distances. 

A  variety  of  automatic  evaluation  techniques  can  be  envisioned  and  will  be  explored  and  compared  with  each 
other  and  with  manual  evaluation  to  find  the  most  accurate  and  practical  method. 

An  accuracy  measure  similar  to  the  average  alignment  error  obtained  manually  can  be  computed  by  splitting 
up  the  two  registered  volumes  into  a  number  of  subvolumes  and  performing  an  iterated  registration  on  each  of  the 
pairs  of  subvolumes,  using  simple  rotate/translate  or  fuU  affine  transformations.  Since  the  subvolumes  are  already 
approximately  aligned,  the  control  points  for  the  second  registration  can  be  placed  automatically  and  do  not 
require  user  interaction.  Assuming  that  the  main  cause  of  error  in  the  initial  registration  is  local  tissue  deformation 
unaccounted  for  by  the  global  deformation  model  applied,  an  iterated  registration  on  small  subvolumes  should 
greatly  increase  the  accuracy  of  the  alignment.  If  the  remaining  error  after  the  second  registration  is  assumed  to 
be  small  compared  to  the  initial  error,  the  registered  subvolumes  can  be  used  as  a  reference  representing  'true' 
alignment.  The  mean  translation  vector  of  all  subvolumes  from  their  positions  after  the  first  registration  to  their 
position  after  the  second  registration  can  be  used  as  an  estimate  of  the  average  alignment  error.  Compared  to  the 
manually  obtained  estimate  it  will  have  the  advantage  of  covering  the  entire  volume  and  being  almost  user 
independent. 

Alternatively,  instead  of  splitting  up  the  volume  and  performing  linear  geometric  transformations  on  each 
subvolume,  a  reference  registration  employing  a  global  non-linear  transformation  can  be  obtained.  The  data  sets 
can  be  registered  with  a  warping  transformation  defined  by  an  increasing  number  of  control  points  until  the 
mutual  information  of  the  final  registration  converges.  Thus  the  mean  warping  distance  of  the  final  transformation 
can  be  used  as  another  estimate  of  the  alignment  error  after  the  initial  registration. 

Both  manual  and  automatic  estimation  of  average  alignment  errors  can  be  validated  in  phantom  scans  using 
tissue  phantoms  with  an  additional,  known  distribution  of  point  reflectors.  The  point  reflectors  are  common 
landmarks  in  the  registered  sets  that  can  be  unambiguously  identified,  thus  enabling  a  very  accurate  manual  error 
estimation  that  can  serve  as  a  gold  standard.  For  the  registration  procedure,  the  point  reflectors  have  to  be  deleted 
from  the  digitized  data  sets  (replaced  by  zero  values),  otherwise  they  would  provide  additional  information  that 
leads  to  better  registrations  than  in  the  tissue  phantom  alone.  The  same  data  sets  (with  the  point  reflectors 
removed)  can  be  used  for  both  manual  and  automatic  error  estimation  and  compared  to  the  gold  standard  based 
on  the  original  data  sets. 

A  problem  with  phantom  studies  is  the  difficulty  in  accurately  simulating  breast  tissue  including  phase 
aberrations  and  tissue  deformability.  Current  phantoms  known  to  the  authors  do  either  not  exhibit  the  aberrations 
present  in  real  breast  tissue,  or  are  much  less  deformable  than  breast  tissue  (30).  As  a  result,  the  registrations 
obtained  in  phantoms  might  be  more  accurate  than  in  vivo  breast  scans  and  therefore  be  unsuited  as  gold 
standard. 

The  accuracy  information  also  helps  to  determine  which  kind  of  mapping  transformation  is  needed  to  provide 
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clinically  useful  registrations.  In  this  study,  only  global  linear  (full  affine)  transformations  were  employed.  In  cases 
with  strong  local  tissue  deformation,  non-linear  warping  transformations  might  be  required  to  keep  the 
registration  error  low.  With  warping,  the  registration  algorithm  may  locally  deform  the  3-D  image  set  in  order  to 
map  deformed  tissues.  Initial  studies  showed  that  including  non-linear  transformations  can  increase  the  mapping 
accuracy  significantly  (31). 

In  future  studies,  the  techniques  developed  here  in  breast  scans  could  also  be  applied  to  scans  of  other  organs. 
Since  phase  aberrations  and  tissue  deformability,  which  introduce  registration  errors,  are  particularly  strong  in 
breast,  registrations  in  liver,  kidney,  prostate  and  other  organs  are  likely  to  be  at  least  of  similar  quality  compared 
to  the  examples  shown  here. 
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Summary:  Two  applications  of  volume  registration  in  3-D  ultrasound  imaging,  extended  volume  imaging 
(EVI)  and  3-D  Spatial  compounding,  are  demonstrated  both  in  phantom  and  clinical  scans.  Extended  volumes 
were  composed  by  obtaining  partly  overlapping  scans,  registering  the  overlapping  volumes,  and  displaying  the 
complete  set  of  volumes  in  their  relative  spatial  orientations  as  determined  by  the  registration  procedure.  3-D 
compound  images  were  created  by  scanning  volumes  from  several  elevational  look  directions,  registering,  and 
averaging  the  different  views.  In  both  applications,  the  volume  registration  worked  accurately  and  provided 
significant  image  improvements. 


INTRODUCTION 

Image  registration,  or  coregistration,  can  be  used  to  combine  several  sets  of  images  obtained 
with  different  imaging  parameters  or  modalities,  from  different  look  directions,  or  at  different 
times.  In  medical  imaging,  it  has  mainly  been  applied  to  magnetic  resonance  imaging.  X-ray 
computed  tomography,  and  tomographic  nuclear  imaging  modalities.  Recently  it  has  been  shown 
that  some  image  registration  techniques  can  also  be  used  to  register  3-D  ultrasound  data  sets  [1- 
3].  In  this  study,  we  demonstrate  two  registration-based  examples  of  image  enhancements  in  3-D 
ultrasound;  Extended  volume  imaging  (EVI)  and  3-D  spatial  compounding.  With  EVI,  one  of 
the  major  limitations  of  high-frequency  ultrasound  scanners,  the  small  field  of  view,  can  be 
overcome.  This  application  is  particularly  important  in  sequential  ultrasound  exams,  e.g.  in 
screening  for  breast  cancer  or  following  caneer  therapy,  where  large  fields  of  view  are  needed  to 
locate  areas  of  change  and  to  make  sure  that  the  same  area  is  covered  in  sequential  scans.  Spatial 
compounding  can  deliver  improved  images  with  reduced  speckle  noise  and  increased  contrast-to- 
noise  ratio.  It  also  reduces  shadowing,  and  creates  more  complete  images  of  connective  tissues 
and  other  specular  reflectors. 


MATERIALS  AND  METHODS 

Since  the  registration  algorithm  we  used  is  fully  explained  in  [4]  ,  we  will  only  briefly 
describe  the  relevant  features  here.  The  software  performs  either  affine,  i.e.  linear,  geometric 
transformations  to  map  one  image  volume  onto  another,  or  non-linear  thin-plate  spline  (TPS) 
warpings.  When  using  full  affine  transformations,  the  user  places  4  initial  control  points  in  each 
data  set.  The  eontrol  points  define  the  approximate  mapping  transformation  from  one  set 
(reference)  onto  another  (homologous).  By  varying  the  position  of  the  control  points,  the 
algorithm  refines  the  transformation  until  the  mutual  information  (MI)  of  the  reference  and  the 


transformed  homologous  set  is  maximized.  The  MI,  defined  as  the  difference  between  the  sum  of 
the  individual  entropies  and  the  joint  entropy  of  two  random  variables,  thus  serves  as  a  cost 
function  in  this  optimization  problem.  For  warping  transformations,  the  user  defines  at  least  one 
more  control  point  and  increases  the  degrees  of  freedom  of  the  registration  transformation  with 
each  additional  control  point.  Note  that  no  pre-processing,  like  cropping  or  speckle  reduction, 
has  to  be  applied  to  the  image  sets  before  registration. 

A  clinical  ultrasound  scanner  (Logiq  700,  GE  Medical  Systems,  Milwaukee,  Wis)  with  an 
llMHz  1.5-dimensional  linear  matrix  array  probe  was  used  for  all  data  acquisition.  With 
electronic  elevational  (normal  to  scan  plane)  focussing,  the  average  axial  and  elevational 
resolution  cell  diameters  were  0.31  mm  and  0.72  mm,  respectively ‘.  The  transducer  was  attached 
to  a  2  degree  of  freedom  position  encoder  that  restrained  the  motion  of  the  probe  to  linear 
elevational  scanning  and  elevational  tilting.  The  position  information  was  used  to  uniformly 
interpolate  and  shear  the  raw  B-scan  image  sets  before  volume  registration.  All  phantom  scans 
were  obtained  in  a  focal  lesion  phantom  (CIRS;  Computerized  Imaging  Reference  Systems, 
Norfolk,  Va).  All  clinical  scans  were  obtained  in  the  female  breast. 

To  create  extended  volume  images,  the  area  of  interest  was  imaged  in  3  or  4  approximately 
parallel,  partly  overlapping  scans  (Fig.  1).  If  3  scans  were  used,  the  central  scan  (B)  served  as  a 
reference  relative  to  which  the  outer  scans  (A,  C)  were  registered  by  maximization  of  the  mutual 
information  in  the  overlaps,  AB  and  BC.  In  cases  where  4  scans  were  combined  (A,  B,  C,  and 
D),  D  was  first  registered  relative  to  C.  The  combined  volume  (C&D)  and  volume  A  were  then 
registered  relative  to  the  global  reference  B  using  the  overlaps  BC  and  AB. 

EVIs  were  obtained  both  in  phantom  scans  and  clinical  breast  scans.  In  phantom  scans,  the 
registration  accuracy  in  the  extended  lateral  direction  was  evaluated  by  comparing  the  true 
position  of  50  spheres  (3mm  diameter)  in  the  phantom  to  the  position  found  in  the  extended 
view.  In  clinical  scans,  the  registered  volumes  were  visually  assessed  and  the  position  of 
common  landmarks  marked  in  each  set.  The  average  registration  error  was  defined  as  the  mean 
distance  between  point  pairs  defining  common  landmarks. 


elevational 


Figure  1:  Extended  Volume  Imaging.  Volumes  A  and  C 
are  registered  relative  to  reference  volume  B. 


Figure  2:  Spatial  compounding  of  volumes  obtained  at 
different  elevational  tilt  angles 


'  A  phantom  was  scanned  axially  and  elevationally  in  an  area  that  produced  fully  developed  speckle.  The  resolution 
cell  diameters  were  defined  as  the  standard  deviation  of  the  Gaussian  fit  to  the  frame-to-frame  correlation  functions. 


For  compound  imaging,  partly  uncorrelated  views  were  obtained  by  changing  the  tilt  angle  of 
the  transducer  before  re-scanning  the  area  of  interest  (Fig.  2).  The  interpolated  and  sheared 
image  sets  were  registered  using  either  full  affine  or  warping  transformations.  The  registered 
data  sets  were  then  log-decompressed,  averaged,  and  recompressed  to  the  same  dynamic  range  as 
the  original  images  (69  dB). 

The  registration  accuracy  in  phantom  scans  was  determined  using  a  semiautomatic  edge 
detection  algorithm.  Given  the  approximate  center  of  a  spherical  hypoechoic  area,  the  algorithm 
finds  the  exact  center  and  the  two  radii  on  which  the  average  signal  increased  10%  and  90%, 
respectively,  of  the  difference  between  the  maximum  signal  level  outside  the  circle  and  the 
minimum  inside.  The  difference  between  the  radii  will  be  referred  to  as  the  boundary  width  of 
the  spheres.  The  accuracy  in  clinical  compound  scans  was  estimated  by  the  same  visual  method 
used  for  clinical  EVI. 


RESULTS  AND  DISCUSSION 

Figure  3  displays  an  example  of  an  axial-lateral  cross  section  through  an  EVI  created  from  4 
partly  (30%,  35%,  and  58%)  overlapping  phantom  scans  registered  with  full  affine 
transformations.  No  averaging  or  other  processing  was  applied  in  the  areas  of  overlap.  The  white 
arrows  indicate  the  display  boundaries  between  the  registered  original  scans.  Note  the  continuous 
appearance  of  specular  reflectors  across  the  boundaries  and  the  spherical  symmetry  of  voids 
placed  on  boundaries.  The  total  width  of  the  cross  section  is  10.6  cm.  The  average  lateral 
deviation  of  the  spheres  in  the  extended  volume  from  their  true  position  is  +0.47%. 

EVI  in  clinical  scans  produced  similar  alignment  quality  as  long  as  the  breast  was  stabilized 
during  the  scans  to  keep  local  tissue  deformation  at  a  minimum  [5].  The  average  distance  of 
common  landmarks  identified  in  the  registered  overlaps  ranged  from  0.8  mm  to  1.4  mm  (peak: 
1.5  to  3.1  mm). 

Correlation  calculations  in  phantom  scans  showed  that  the  change  in  tilt  angle  needed  to 
create  uncorrelated  images  is  4  to  6  degrees.  In  compounding  phantom  scans  obtained  at 
differential  scan  angles  of  4  degrees,  the  signal-to-speckle  noise  ratio  (SNR)  increased  10%  to 
20%  less  than  the  theoretical  vN  dependence  for  N  compounded,  uncorrelated  images.  The 
average  misregistration  in  phantom  scans  using  full  affine  transformations  was  0.21±0.02  mm 
and  did  not  vary  significantly  with  tilt  angle.  As  a  result  of  misregistrations,  the  average 
boundary  width  of  the  spheres  in  a  phantom  image  compounded  from  5  scans  increased  20% 
compared  to  the  uncompounded  image.  In  the  same  compound  image,  the  SNR  increased  95%. 


Figure  3:  Axial-lateral  cross  section  of  an  extended  volume  created  from  4  partly  overlapping  phantom  scans. 


Figure  4  shows  an  example  of  an  uncompounded  clinical  image  in  breast  (a),  and  the  same 
image  compounded  from  5  scans  at  0°,  6.4°,  12.8°,  -5.1°,  and  -10.2°,  using  11  control  point 
warping  transformations  (b).  Statistical  analysis  showed  an  almost  ideal  o^Vn  increase  in  SNR 
and  CNR  (contrast-to-noise  ratio)  for  N  compounded  images.  Note  the  overall  reduction  in 
granularity  and  the  better  delineation  of  connective  tissues  (arrows). 

The  same  data  sets  were  also  registered  using  full  affine  transformations.  Comparing  the 
warped  data  sets  to  the  linearly  transformed  sets  we  found  an  increase  in  average  estimated 
misregistration  from  0.42±0.10  mm  to  0.75±0.12  mm,  suggesting  that  warping  can  correct  for 
local  tissue  deformation  that  is  unaccounted  for  by  global  linear  transformations. 

We  conclude  that  image  registration  is  a  promising  tool  in  3-D  ultrasound,  allowing 
applications  such  as  accurate  extended  volume  imaging  and  3-D  compounding.  Using  image- 
based  registration  with  non-linear  mapping  transformations  prior  to  compounding  offers  the 
unique  potential  to  correct  for  tissue  motion  and  refraction  artifacts,  effects  that  can  greatly 
distort  image  quality  in  other  compounding  approaches. 


Figure  4:  (a)  Uncompounded  image  from  reference  (b)  Same  image  compounded  from  5  scans  at  0°,  6.4°, 
scan.  12.8°, -5.1°,  and -10.2°. 
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Abstract: 

Diverse  developments  of  3D  ultrasound  imaging  of  the  breast  and  breast  lesion  vascularity  have 
been  undertaken,  and  increasing  promise  of  characterization  of  benign  versus  malignant  lesions  has 
been  reported.  As  a  notable  example,  we  recently  studied  a  moderate  number  (38)  of  women 
scheduled  for  breast  biopsy  and  found  particular  value  in  Speed  Weighted  pixel  Density  (SWD). 

The  technique  uses  a  position-encoding  apparatus  developed  by  our  group  to  gather  parallel 
frequency-shift  Doppler  images  (f-CDI)  of  each  volunteer  subject.  A  radiologist  then  identifies  each 
suspicious  mass  with  a  3D  ellipsoid,  dynamically  positioned  and  shaped  within  each  reconstructed  f- 
CDI  volume.  The  maximum  vascularity  (SWDmax)  is  determined  from  regions  in  and  around  the 
lesion  and  is  then  assessed  in  conjunction  with  gray  scale  (GS)  characteristics,  which  are  ranked  and 
averaged.  Pathologic  findings  of  this  particular  group  (18  benign,  20  malignant)  were  used  to 
evaluate  our  diagnostic  accuracy,  as  demonstrated  by  receiver  operating  characteristics  (ROC)  for 
SWD  and  GS  as  well  as  combinations  of  these,  with  Age,  which  provided  SWD-Age,  SWD-GS,  and 
SWD-Age-GS  indices.  Performance  of  SWD  was  comparable  to  that  of  the  subjective  GS,  as 
indicated  by  the  areas  under  the  ROC  curves,  roughly  equal  to  0.9  in  both  cases,  where  1.0  describes 
the  perfect  discriminator.  The  two-variable  indices  enhanced  discrimination,  while  SWD-Age-GS 
yielded  100%  cancer  discrimination.  The  particularly  attractive  SWD-Age  results,  given  the 
objectivity  of  both  variables,  and  the  remarkable  three-variable  results  in  this  patient  pool  suggest 
that  quantitative  vascularity  measurements  can  considerably  improve  malignant  tissue 
characterization  beyond  subjective  GS  evaluation  alone. 

Introduction: 

The  relatively  non-specific  nature  of  mammography  [1]  in  the  early  detection  of  breast  cancer  has 
led  to  a  variety  of  classification  schemes  to  enhance  diagnoses  [2].  Other  imaging  modalities  such 
as  ultrasound  have  also  been  employed  to  reduce  needless  biopsies  [3,4].  Based  on  the  potential 
benefits  of  combining  3D  sampling  and  quantitative  vascular  assessments,  a  preliminary  study  was 
performed  by  our  group  to  evaluate  which  color  Doppler  ultrasound  signals  might  provide 
discrimination  of  breast  cancer  from  benign  masses  [5].  In  a  subsequent  study,  a  particular 
vascularity  measure.  Speed  Weighted  pixel  Density  (SWD)  was  measured  in  each  of  4  regions  of 
interest  in  and  around  suspicious  breast  masses  and  compared  favorably  to  standard  gray  scale 
criteria  [6].  We  present  here  results  from  the  ongoing  investigation  of  this  promising  discrimination 
technique. 

Methods: 

Our  subject  pool  is  comprised  of  volunteer  patients  with  suspicious  breast  masses  scheduled  for 
biopsy.  The  38  women  presented  here  were  evaluated  using  a  GE  Logiq  700  and  M12  linear  matrix 
array  scanhead  (6  MHz  Doppler,  9  MHz  gray  scale).  Parallel  frequency-shift  color  Doppler  image 
(f-CDI)  slices  are  obtained  at  known  intervals  by  translating  the  transducer  while  acquiring  distance 
measurements  from  a  linear  encoder  as  shown  in  Figure  1  above.  These  image  sets  are  subsequently 
cropped  and  reconstructed  into  3D  ultrasound  volumes  using  a  DEC  workstation,  AV S/Express,  and 
in-house  processing  software.  A  radiologist  identifies  each  suspicious  mass  with  a  3D  ellipsoid, 
dynamically  positioned  and  shaped  within  each  reconstructed  f-CDI  volume.  Speed  Weighted  pixel 
Density  (SWD),  defined  as  the  mean  velocity  times  the  colored  pixel  density,  is  calculated  in  each  of 
the  four  volumes  of  interest  shown  in  Figure  2.  The  maximum  vascularity  index  (SWDmax)  is  the 


maximum  of  these  four  calculations.  An  example  of  the  radiologist’s  ellipsoidal  definition  is  shown 
in  Figure  3.  Gray  scale  (GS)  characteristics  of  each  mass  are  also  evaluated,  ranked  on  a  scale  of  1 
to  5  (low  to  high  suspicion),  and  averaged.  These  include  margin  smoothness,  margin  visibility, 
homogeneity,  attenuation,  echogenicity,  shape,  and  height.  Linear  combinations  of  these  values  of 
SWDmax,  GS,  and  patient  age  give  rise  to  a  variety  of  indices  and  form  the  basis  of  the 
differentiation  of  malignant  from  benign  lesions  presented  here.  Pathologic  findings  are  used  as 
“truth”  data  in  classifying  a  given  case  as  benign  or  malignant.  Discrimination  thresholds  are 
determined  using  a  linear  Baysian  classifier  for  the  two-variable  and  three- variable  cases.  In 
addition,  thresholds  are  determined  to  minimize  false-negative  diagnoses.  Finally,  maximum 
likelihood  estimates  of  single  binormal  ROC  curves  for  the  indices  are  calculated  to  determine  their 
comparative  performance.  Shown  here  are  comparative  results  of  SWD  and  GS  as  well  as  the 
multi-variable  indices  SWD- Age,  SWD-GS,  and  SWD-Age-GS. 

Results: 

As  shown  in  Figure  3,  vascularity  is  easily  visualized  from  the  3D  reconstructed  ultrasound  volume. 
Not  apparent  in  a  static  picture  is  the  ease  with  which  the  entire  volume  can  be  inspected  in  three 
dimensions.  The  ellipsoid  shown  was  produced  during  one  such  session.  More  quantitatively. 
Figure  4  shows  a  relatively  “raw”  representation  of  our  historically  most  promising  measure  of 
vascularity,  log(SWDmax),  versus  patient  age.  One  can  appreciate  the  discriminating  ability  of 
SWD  alone  by  envisioning  a  horizontal  line  on  the  same  plot.  Also  of  note  is  the  decrease  in 
vascularity  with  age  (particularly  in  the  benign  cases),  which  makes  the  two-variable  index 
somewhat  intuitive.  The  diagonal  line  represents  a  Baysian  discrimination  line,  equally  weighing 
the  cost  of  false-positive  and  false-negative  diagnoses.  Compressing  these  data  into  a  single  index 
yields  the  graph  shown  in  Figure  5.  Note  the  reasonable  specificity  (True  Negative  Fraction)  when 
the  threshold  is  lowered  to  produce  a  sensitivity  of  100%.  Using  a  3D  linear  Baysian  discrimination 
scheme  (which  produces  a  threshold  plane),  the  average  gray  scale  rating  (GS)  is  included  in  a 
similar  representation  of  the  three-variable  index  SWD-Age-GS  shown  in  Figure  6.  A  classification 
scheme  slightly  weighted  (2:1)  to  reduce  the  risk  of  false  negatives  produces  a  threshold 
discriminator  able  to  completely  differentiate  between  malignant  and  benign  cases.  Figure  7 
summarizes  the  diagnostic  performance  of  the  various  indices  with  maximum  likelihood  estimates  of 
receiver  operating  characteristic  (ROC)  curves  for  the  indices  shown  in  Figures  5  and  6  as  well  as 
the  SWD-GS  index  and  the  SWD  and  GS  variables  alone.  As  indicated  by  the  area  (Az)  under  each 
ROC  curve,  the  performance  of  SWD  was  comparable  to  that  of  the  subjective  GS  [Az(SWD)=0.90; 
Az(GS)=0.89].  Discrimination  is  enhanced  with  a  two  variable  measure,  as  demonstrated  by  the 
SWD- Age  and  SWD-GS  results  [Az(SWD-Age)=0.96;  Az(SWD-GS)=0.97],  while  SWD-Age-GS 
yielded  100%  discrimination  of  cancer.  Evaluation  of  the  statistical  significance  of  these  recent 
findings  is  currently  underway.  (Comment:  1  would  think  that  there  would  be  some  way  to  calculate 
the  relative  significance  (confidence)  of  a  given  discrimination  value  (90%  for  one  variable  vs  100% 
for  combining  three  with  a  patient  population  of  38)  based  on  the  number  of  variable  added  and  the 
number  of  patients  included  in  the  study.  Unfortunately  1  am  not  sure  how  this  is  done  statistically. 
Perhaps  Jon  Rubin  or  Paul  would  know  or  we  can  go  to  the  statistical  core  with  it.) 

Conclusions: 

Vascularity  in  and  around  suspicious  breast  lesions  can  be  quantified  and  used  as  an  adjunct  measure 
to  current  diagnostic  tests.  These  vascular  measures  can  be  easily  combined  in  a  linear  fashion  with 
other  indicators  to  produce  useful  indices.  Since  the  performance  of  the  combined  SWD-Age 


measure  is  comparable  to  the  SWD-GS  index,  it  is  a  particularly  attractive,  given  the  objectivity  of 
both  the  SWD  and  Age  variables.  Combining  the  SWD-Age  index  with  a  subjective  gray  scale 
rating  yields  remarkable  results,  at  least  in  this  particular  patient  pool,  and  suggests  that  quantitative 
vascularity  measurements  can  considerably  improve  malignant  tissue  characterization. 
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Figure  Captions: 


Figure  1.  Scanning  apparatus.  The  hand-controlled  transducer  device  is  designed  to  provide  guided 
trajectory  of  the  transducer  while  measuring  linear  position  and  rotation  angle.  At  the  base,  a  linear 
position  encoder  is  fastened  behind  a  small  linear  bearing  which  enables  motion  in  the  x-axis, 
perpendicular  to  the  transducer.  An  output  signal  across  the  encoder  is  acquired  by  our  computer 
interface  for  position  registration  (resolution  =18  microns).  A  light  weight  rotary  optical  encoder 
provides  angular  measurement  with  a  resolution  of  0.04  degrees.  The  patient  is  scanned  through  the 
window  in  the  base  plate. 

Figure  2.  Regions  of  interest  in  and  around  the  suspicious  mass.  A  3  mm  shell  is  computer  generated 
around  the  radiologist’s  ellipsoidal  estimate  of  the  mass.  Our  quantitative  values  are  calculated  in  foui 
regions:  (a)*  the  proximal  half  of  the  radiologist’s  ellipsoid,  (b)*  the  entire  ellipsoid  of  the  radiologist, 
(c)*  the  proximal  half  of  the  shell,  and  (d)*  the  entire  3  mm  shell. 

Figure  3.  Example  of  a  radiologist  defined  ellipsoid.  Three  ultrasound  image  planes  are  shown  in  the 
example,  highlighted  with  3D  rendered  vascularity  in  a  55  year  old  patient  with  invasive  ductal 
carcinoma  with  lymphatic  invasion.  Two  of  the  image  planes  are  reconstructed  from  the  3D  volume. 
The  vascularity  is  apparent,  mostly  in  the  immediate  periphery  of  the  ellipsoid. 

Figure  4.  Discrimination  of  benign  (*)  and  malignant  (*)  breast  tumors  for  38  cases  studied:  18  benig 
(mean  age  46),  20  malignant  (mean  age  56).  The  plot  depicts  one  of  our  quantitative  vascularity 
measures,  log  of  the  maximum  value  of  Speed  Weighted  pixel  Density  (SWD),  versus  patient  age. 
Using  the  slope  (0.051)  of  the  discrimination  line  determined  by  a  linear  Baysian  classifier,  a  combinei 
SWD- Age  index  was  calculated  for  each  patient  and  plotted  in  Figure  5. 

Figure  5.  Combined  SWD-Age  Index.  Points  are  spread  horizontally  for  clarity.  The  upper  threshold 
line  shown  represents  the  value  of  log(SWDmax)  at  Age=0  of  the  unweighted  discrimination  line  of 
Figure  4.  This  threshold  yields  a  reasonable  true  positive  fraction  (TPF)  and  true  negative  fraction 
(TNF)  of  0.90  and  0.94,  respectively.  Requiring  a  TPF  of  1.00  reduces  the  TNF  to  0.56.  A  similar 
approach,  including  the  average  grayscale  rating  (GS),  produces  the  SWD-Age-GS  index  shown  in 
Figure  6. 

Figure  6.  Discrimination  Index  including  one  subjective  measure,  grayscale  rating  (GS).  As  depicted 
with  the  upper  horizontal  line,  an  unweighted  discriminator  provides  excellent  sensitivity  (90%)  and 
specificity  (100%).  Although  these  data  represent  only  38  patients,  the  promising  result  of  complete 
discrimination  (lower  threshold  line)  can  be  achieved  with  a  Baysian  classification  scheme  slightly 
weighted  (2:1)  to  reduce  the  risk  of  false  negatives. 

Figure  7.  Diagnostic  performance  of  SWD,  GS,  and  Age.  The  curves  shown  are  maximum  likelihood 
estimates  of  ROC  curves  for  the  indices  shown  in  Figures  5  and  6  as  well  as  a  SWD-GS  index  and  the 
SWD  and  GS  variables  alone.  As  indicated  by  the  area  under  each  curve  (Az),  the  solo  performance  o 
our  quantitative,  objective,  measure  of  vascularity  (SWD)  is  comparable  to  that  of  the  subjective 
grayscale  rating.  Two-variable  indices  enhance  discrimination,  as  demonstrated  by  the  results  of  SWE 
GS  and  SWD-Age.  Adding  one  subjective  variable,  GS,  to  the  SWD-Age  index  yielded  remarkable 
results  (at  least  in  this  particular  patient  pool),  providing  %100  cancer  discrimination. 
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trajectory  of  the  transducer  while  measuring  linear  position  and  rotation  angle.  At  the  base,  a  linear 
position  encoder  is  fastened  behind  a  small  linear  bearing  that  enables  motion  in  the  x-axis, 
perpendicular  to  the  transducer.  An  output  signal  across  the  encoder  is  acquired  by  our  computer 
interface  for  position  registration  (resolution  =18  microns).  A  light  weight  rotary  optical  encoder 
provides  angular  measurement  with  a  resolution  of  0.04  degrees.  The  patient  is  scanned  through  the 
window  in  the  base  plate. 


Figure  2.  Regions  of  interest  in  and  around  the  suspicious  mass.  A  3  mm  shell  is  computer 
generated  around  the  radiologist’s  ellipsoidal  estimate  of  the  mass.  Our  quantitative  values  are 
calculated  in  four  regions:  (a)*  the  proximal  half  of  the  radiologist’s  ellipsoid,  (b)*  the  entire 
ellipsoid  of  the  radiologist,  (c)*  the  proximal  half  of  the  shell,  and  (d)*  the  entire  3  mm  shell. 


Figure  3.  Example  of  a  radiologist  defined  ellipsoid.  Three  ultrasound  image  planes  are  shown  in 
the  example,  highlighted  with  3D  rendered  vascularity  in  a  55  year  old  patient  with  invasive  ductal 
carcinoma  with  lymphatic  invasion.  Two  of  the  image  planes  are  reconstructed  from  the  3D  volume. 
The  vascularity  is  apparent,  mostly  in  the  immediate  periphery  of  the  ellipsoid. 
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Figure  4.  Discrimination  of  benign  (*)  and  malignant  (*)  breast  tumors  for  38  cases  studied:  18 
benign  (mean  age  46),  20  malignant  (mean  age  56).  The  plot  depicts  one  of  our  quantitative 
vascularity  measures,  log  of  the  maximum  value  of  Speed  Weighted  pixel  Density  (SWD),  versus 
patient  age.  Using  the  slope  (0.051)  of  the  discrimination  line  determined  by  a  linear  Baysian 
classifier,  a  combined  SWD- Age  index  was  calculated  for  each  patient  and  plotted  in  Figure  5. 
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Figure  5.  Combined  SWD-Age  Index.  Points  are  spread  horizontally  for  clarity.  The  upper 
threshold  line  shown  represents  the  value  of  log(SWDmax)  at  Age=0  of  the  unweighted 
discrimination  line  of  Figure  4.  This  threshold  yields  a  reasonable  true  positive  fraction  (TPF)  and 
true  negative  fraction  (TNF)  of  0.90  and  0.94,  respectively.  Requiring  a  TPF  of  1.00  reduces  the 
TNF  to  0.56.  A  similar  approach,  including  the  average  grayscale  rating  (GS),  produces  the  SWD- 
Age-GS  index  shown  in  Figure  6. 
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Figure  6.  Discrimination  Index  including  one  subjective  measure,  grayscale  rating  (GS).  As 
depicted  with  the  upper  horizontal  line,  an  unweighted  discriminator  provides  excellent  sensitivity 
(90%)  and  specificity  (100%).  Although  these  data  represent  only  38  patients,  the  promising  result  of 
complete  discrimination  (lower  threshold  line)  can  be  achieved  with  a  Baysian  classification  scheme 
slightly  weighted  (2:1)  to  reduce  the  risk  of  false  negatives. 
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Figure  7.  Diagnostic  performance  of  SWD,  GS,  and  Age.  The  curves  shown  are  maximum 
likelihood  estimates  of  ROC  curves  for  the  indices  shown  in  Figures  5  and  6  as  well  as  a  SWD-GS 
index  and  the  SWD  and  GS  variables  alone.  As  indicated  by  the  area  under  each  curve  (Az),  the 
solo  performance  of  our  quantitative,  objective,  measure  of  vascularity  (SWD)  is  comparable  to  that 
of  the  subjective  grayscale  rating.  Two-variable  indices  enhance  discrimination,  as  demonstrated  by 
the  results  of  SWD-GS  and  SWD-Age.  Adding  one  subjective  variable,  GS,  to  the  SWD-Age  index 
yielded  remarkable  results  (at  least  in  this  particular  patient  pool),  providing  %100  cancer 
discrimination. 
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